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ABSTRACT 



This ESEA Title III document synthesizes three 
reports from regional workshops held between September 1969 and 
January 1970. The topics concern issues discussed at these workshops. 
The first two papers, reflecting r consensus of workshop 
participants, indicate the essential conditions that should exist 
within State educational agencies to facilitate creative educational 
changes, and provide recommendations that State agencies can use to 
improve ESEA Title III programs. The third paper describes strategies 
for assessing educational needs in the cognitive, affective, and 
psychomotor domains. The document also includes presentations made by 
speakers at the various conferences and an evaluation based on 
questionnaires used before and after each conference. An appendix 
summarizes conference participation by State. (DE) 
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This report is a synthesis of three others. As such, it has evolved 
from the grass roots. Into it has gone summary recommendations from three Title 
III ESEA regional meetings held between November 17 and December 12, 1969. Into 
it has also gone a sharing of problems and ideas relating to certain key educa- 
tional issues by participants from each of the fifty states, the District of 
Columbia, the Virgin Islands, Guam, Puerto Rico, the Trust Territory of the Pa- 
cific, American Samoa, and the Bureau of Indian Affairs. These issues are unique 
ly the concern of the U. S. Office of Education, Title III State coordinators. 
Title III State advisory committees, and Chief State School Officers. It was 
for these educators and school patrons that the conferences were designed. 

It should be stated at this point that the regional meetings, in their 
entirety, could be referred to as workshops. That is what they were. They are 
occasionally referred to, here, as conferences merely for convenience in dis- 
tinguishing between them and the separate workshops held on each of three days 
during their five-day schedules of activity. 

Planned around the theme of 'facilitating the Creative Educational 
Change Process Through Effective Management," the meetings were hosted by the 
ESEA Title III offices of the States of Florida, Pennsylvania, and Arizona. The 
first conference was held at Fort Lauderdale on November 17-21 for midwe stern 
and southern States; the second, at Philadelphia on December 1-5 for northeastern 
States; and the third, at Phoenix on December S-13 for far western and south- 
western States. 

Major issues explored were concerned with the following: (a) the es- 

sential conditions which should exist in State educational agencies to facilitate 
creative change; (b) the understanding and competency required in the operation 
and management of Title III programs to provide educational accountability; and 
(c) the strategies which could contribute to State staff development, educational 
needs assessmpnt, and evaluation and dissemination to complete the FY 1970 Title 
III State Plan requirements,. 

In addition to exploration of these issues, the conferences had several 
other purposes. They included: providing orientation for new Title III coordina 

tors and new State advisory council members, providing opportunities to discuss 
individual State problems, and increasing understanding and competency in the 
development of strategies and techniques for evaluation and dissemination, gener- 
ally. These latter three objectives were accomplished on the first day of each 
regional meeting through presentations made by Dr. 0. Ray Warner, Chief, State 
Plans for Educational Innovation Branch, U. S. Office of Education, and his staff 

Back of the commences , of course, lies the recent history of Title 
III. Primary responsibility for this section of the Elementary and Secondary 
Education Act *>f 1965 — often referred to as PACE (Projects to Advance Creativity 
in Education) — passed from the U. S. Office of Education to State education agen- 
cies, as a result of amendments to the Act, on July 1, 1968. At that time, USOE 
organized to provide leadership for State planning through the State Plans for 
Educational Innovation Branch of the Bureau of Elementary and Secondary Education 



Through the planning and coordination of this branch, conferences 
were held in mid-1968 in each of Title Ill's three regions to facilitate 
State management in the initiation and implementation of innovative educa- 
tional programs. These conferences earned unqualified approval by partici- 
pants. In the fall of 1969, accordingly, it was deemed important to have 
additional meetings for discussion of emerging problems and concerns and 
for analysis — after some eighteen months of State operation — of key issues. 

A planning committee, made up of representatives from various States, the 1 
met on September 1, 1969, to draw up directions for the meetings. This 
committee gave the State Plans for Educational Innovation Branch the charge 
to develop conference objectives and format. Responsibility for overall 
planning was accepted by Dr. 0. Ray Warner, Chief of that branch, and for 
supporting activities, including evaluation of the meetings, by Dr. Norman 
E. Hearn, Chief, Program Analysis and Dissemination Branch. Host states 
for the regional conferences were Florida for Region II, Pennsylvania for 
Region I, and Arizona for Region III. 

The mechanics of the regional conferences called for dividing the 
total meeting memberships into small workshop groups of six to ten persons 
to allow the greatest possible involvement and communication. Three days 
of each conference were devoted to consideration of the three major issues, 
one issue each day, by the workshop groups. The groups, as far as possible, 
represented a geographical and occupational mix. 

At the end of each workshop, each group chairman turned in the 
results of his group's deliberations to a report chairman — one assigned 
to each of the three major issues — whose duty it wa3, in turn, to report 
to the general conference membership on the last day of the conference a 
consensus of the workshop deliberations * 

In January, 1970, the report chairmen convened in Washington to 
consolidate reports developed at the workshops and to develop papers which 
would reflect the thinking of the States on the various issues involved, 
particularly on how these issues related to Title III ESEA. In so doing, 
the report chairmen worked toward objectives embodying the issues, already 
alluded to in this introduction, as follows: 

(a) development of a paper indicating the essential and desir- 
able conditions, as determined by a consensus of the conference partici- 
pants, which should exist in State educational agencies to bring about 
creative educational changes in the States; 

(b) development of a papers reflecting a consensus of workshop 
participants; which can be used by Sfate educational agencies to improve 
Title III E^EA programs through educational accountability; 

(c) development of a ppper which describes strategies for as- 
sessing educational needs in the cognitive, affective, and psychomotor 
domains. 



In this report, the sections immediately following are concerned 
with these objectives. Each section reporting the results of workshop 
deliberations begins with a list of recommendations derived from workshop 
co\; census. 
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This report also includes presentations made by speakers at the 
various conferences and an evaluation based on pre- and post-conference 
questionnaires. Finally, it includes an appendix which summarizes confer- 
ence participation by each of the States. 
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WorkshoD 



Objective: To develop a paper which indicates the essential and desirable con- 

ditions, as determined by a consensus of the workshop participants, 
which should exist in State educational agencies to bring about 
creative educational changes in the State. 

RECOMMENDATIONS 

1. State educational agencies should encourage support of innovative ideas or 
reinforce and expand innovative efforts through stipends, grants, teacher 
exchanges, etc. 

2. State educational agencies should assume the initiative in needs assessment, 
not only at their own level but at the local agency level as well. 

3. State educational agencies should create a management information system for 
their own use. 

4. State educational agencies should develop flexible staffing which will be 
representative of the skills and specializations needed in educational change 

5. State educational agencies should require staff development programs to im- 
prove leadership capabilities. 

6. State educational agencies should develop special programs to improve evalua- 
tion and dissemination at both the state and local level by 

a* providing staff development in the areas of evaluation and dissemina 
tion, 

b. focusing on the use of evaluative data to improve funding operations 

c. coordinating State efforts in evaluation and dissemination, 

d. focusing on evaluation and dissemination as aids in the decision- 
making process. 

7. State educational agencies should encourage or require continuing staff de- 
velopment program:* which are focused on local educational agency needs. 

8. State educational agencies should develop a planning unit which will 

a. evaluate its own organization and function in relation to its varied 
roles, 

b. improve cooperation and coordination between State and local agen- 
cies through use of resources of both. 



9. State educational agencies should develop programs of information exchange 
and involvement with communities. 

10. State educational agencies should fund projects which are designed to de- 
velop better school-community involvement. 

11. State educational agencies should give local educational agencies support 
for development of successful models for visitation, dissemination, and 
demonstration. 

12. State educational agencies should seek cooperative involvement with colleges 
and universities to promote improved teacher education programs and research 
which is sensitive uo current educational needs. 

Any analysis of the functions performed by State educational agencies 
must rest on two assumptions : First, that despite the operational distance be- 
tween the State agency and the local school, these two institutions share a pri- 
mary concern — the student and his educational welfare. And, second, that the 
State educational agency can no better escape today* s rapid and pervasive social 
and technological change than can other public organizations; accordingly, rather 
than wait to be overtaken, it should instead initiate and manage change in the 
most effective manner. 

From these assumptions, development of State education department capa- 
bility for functions associated with the change process begins with a systematic 
identification of desired conditions for change as well as of constraints and 
barriers thereto. It then continues with identification and implementation of the 
processes designed to facilitate the department 1 s role as a change agent. 

Recommendations for facilitating change have been given in the immedi- 
ately preceding* twelve-point list. As an assembly of spokes must be attached 
to a hub, the twelve points, however, must be fixed to a center of prerequisite 
characteristics on the part of the State agency. These include comprehensive, 
adaptive, and open systems, designed for equal educational opportunities, human- 
istic rather than mechanistic, and flexible enough to allow priority changes 
within variable time schedules. 

The process of identifying barriers to desirable change begins with 
recognition of typical resistance to change as change. It is easier and more 
comfortable to follow traditional, hence fixed, procedures. It is no accident 
that State educational agencies today, whether striving to cope with change or 
not, are burdened with an image which outlines them as passive, supervisory, and 
regulatory agencies, exerting little if any leadership. On the other hand, in 
some instances where leadership has been attempted, it has been resisted by 
local educational agencies and, on occasion, by segments of the State agency, 
itself. 



Turning to ether barriers, inefficient educational management systems 
at both the State and local levels constitute part of current conditions which 
must be corrected to effectively and efficiently institutionalize change. Frag- 
mentation in the organizational structure of State agenciee leads to duplication 
of function and poor interagency communication. Lack of specialists trained in 
the myriad tasks of facilitating change has led to considerable running in place 
and trials marked by considerable error. The growing interest and effort in 
Title III, ESEA, staff development programs offers a model which other elements 

O 
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of State and local agencies could profitably replicate. 

Research and development activities have not as yet been widely adopted 
or adequately supported. Here, again, aspects of research and development activi- 
ties demonstrated by Title III suggest strengths which could be realized by other 
sections of State and local education agencies. 

The lack of fully developed needs assessments at State and local levels 
prevents construction of context information bases needed, in turn, to provide 
objective evidence in setting up specific goals and procedures for effecting 
change. 



The unequal distribution of educational resources, as evidenced by so- 
called rich or poor districts, inhibits widespread change. It is possible that 
State agencies could use districts which do hive the necessary resources for re- 
search and development as vehicles for bringing about Statewide change. 

Related to inadequate resources is the meager skill demonstrated by 
the educational community in working effectively with the community-at-large to 
create support for bringing about desirable change. Initial steps in correcting 
this situation, however, have baen taken by Title III advisory groups and coun- 
cils. Such steps should be followed by other members of the educational com- 
munity. 



Dissemination efforts are often deficient or misdirected, by-passing 
many relevant target audiences. The awakened interest in audience identifica- 
tion, user analysis, and further definition of local State and Federal roles in 
disseminating appropriate information regarding change should provide impetus 
for designing dissemination systems which will remove this constraint. 

In the identification of constraints, it may escape notice that vari- 
ous barriers could be converted into facilitators of change. Into this category 
falls the one-dimensional sense of responsibility characteristic of many local 
districts and a consequent lack of district coordination and cooperation. The 
strong self-motivation often characteristic of local districts, however, is a 
strength which State agencies, with appropriate attitudes and procedures, may 
build upon. Also falling into this category, but perhaps less clearly so, are the 
isolated pockets of change within an educational system, whose very isolation may 
be seen as evidence of poor implementation systemwide. Once an entire system is 
involved in the change process, however, any such individual areas of change may 
act as spurs tc speedier realization of desired ends. 

Conservatism both imposes constraints and serves as a strength in 
bringing about chanf e. In restrictive legislation and insufficient financial 
support, conservat* ,m sets up a barrier. In guarding against gross, disruptive 
change, it adds stability to the change process. 

Social conflict often prevents long range educational planning and 
forces educators to devote their time and energy to putting out fires rather 
than to designing and implementing the very plans which will mediate conflict. 

At the same time, however, conflict holds up a mirror to problems for which 
solutions must be found. 

Another barrier related to social conditions is the increased mobility 
of our population, which acts to disrupt orderly and lasting change. Mobility 
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of peoples, however, can also assist in the spread of innovative ideas if properly 
dealt with. 



In summary, commitment to educational improvement by State agencies 
should include a strategy for organizing various agency components in an orderly 
approach to constructive change. The first step in the change process must be 
an assessment of current programs measured against performance standards that 
serve as realistic educational goals. The functions of the State educational 
agency must necessarily include a mechanism that will generate the kind of in- 
formation upon which decisions may be based. This will provide a foundation for 
designing program strategies related to needs that have been identified as in- 
adequately served by current programs. A State education agency must hold it- 
self accountable for improving the system of education for which it is respon- 
sible. 



Workshop II 



Objective: To develop a paper, reflecting a consensus of the workshop partici- 
pants, which can be used by the State educational agencies to im- 
pro e the ESEA Title III program through educational accountability. 

RECOMMENDATIONS 



Measurement 



1. State educational agencies should require that ob jectives--whether in cogni- 
tive, affective, or psychomotor domains, as dictated by the nature of a proj- 
ect--be stated in measurable terms, with performance specifications. 

2. State educational agencies should require that appropriate measurement pro- 
cedures and instruments be employed to determine the degree to which each 
objective is accomplished and that such procedures and measurements not be 
restricted to standardized tests but include subjective judgments, logs of 
critical events, descriptive incidents, follow-up studies, check lists, etc. 

3. State educational agencies should require continual evaluation of project 
activities so that program modifications can be designed for maximum effective- 
ness. 

4. State educational agencies should require annual outside educational audits 
of all projects and agreement between internal and external evaluation re- 
sults. 

Management 



5. State educational agencies should require that project personnel be adequate- 
ly trained to perform their tasks and thac they devote their contractual time 
and effort to project activities. 

6. State educational agencies should require a clearly projected time line of 
activities and results. 

7. State educational agencies should require well defined channels of communi- 
cation with boards of education, school administrations, teachers, and lay 
publics . 

8. State educational agencies should require evidence of commitment to the sup- 
port of successful practices by grantees. 



General Implementation 



9* State educational agencies and grantees should jointly implement procedures 
which will insure that practices which are demonstrably successful will be 
continued not only in the districts of the grantees in question but in other 
school districts within the State. 

For the purpose of the workshops, accountability was defined as that 
act whereby any school agent accepting a grant is held responsible for performing 
according to the terms of the grant. This definition implies achieving the pro- 
posed objectives within the time period or periods specified, within the budget 
limitations designated, and according to the standards established. 

While not stated in the integrated report on accountability, implicit 
in the individual conference reports were varying degrees of reservation, an ex- 
pressed need for greater understanding of the term and its implications, and a 
call for caution in procedures leading to adoption of this currently provocative 
concept. 



Accountability accompanied by a determination of cost effectiveness 
was seen as appropriate for conventional programs but as possibly posing a threat 
to innovative programs characteristic of those under Title III. This would be 
true, at least, where there is a generally fuzzy or invalid notion about the nacure 
of innovation and research. Concern was also expressed over application of the 
concept of accountability to programs having to do with the affective domain and 
over increased dehumanization in education in general. 

At the same time, there was general acceptance of the basic idea and 
recognition that procedures for educational accountability are developing in all 
areas of education. It was acknowledged that educational goals have often been 
stated in vague, global terms and educational evaluations made only through stand- 
ardized tests or subjective judgments. As stated in one conference report, "The 
need to define precise objectives, to be attained by specific action, has typical- 
ly gone unrecognized." 

The unrecognized need for accountability to education* s clients, ac- 
cording to some conference participants, may be the culprit responsible for in- 
equality of educational opportunity, for frequency of bond issue failure, for 
inadequate preparation for the world of work by large numbers of America's youth, 
and even, perhaps, for the widening gulf between students and schools. Statis- 
tical reporting of per pupil expenditures in terms of ADA, student- teacher ratio, 
and teachers 1 salaries were seen as not significantly related to educational 
achievement . 

In exploring the implications of educational accountability, conference 
members examined a list of eleven performance areas, including: 

1. analysis of needs 

2. statement of measurable performance objectives 

3. selection and assignment of staff 

4. project management 

5. selection and use of resources 

6. logistical or PERT planning 

7. program operation 

8 0 evaluation of outcomes or product against performance criteria 



memb< 
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9. evaluation of operational process and project management 

10, cost effectiveness determination 

11. demonstration and dissemination 

The list, it was said, is generally common to State Title III staffs, 
with the notev;orthy exception of cost analysis determination. Since State super- 
vision for the past year, however, has been requiring measurable objectives and 
since projects characterized by global objectives can ordinarily be expected to 
phase out this coming June, it seems reasonable to predict that formal account- 
ability will be a component of Title III programs by June, 1971. 

In planning for accountability, State educational agencies should es- 
tablish, for the operation of all projects, guidelines and criteria which fall 
into three categories: measurement, management, and general implementation. 

Specific steps under these categories were outlined in recommendations listed on 
preceding pages. A more general but important recommendation calls for clearly 
defined lines of administration of Title III established in the U. S. Office of 
Education, in State education agencies , and in local projects. Such a step 
would safeguard accountability at the various operational levels of Title III 
programs . 



In summary, there was general agreement that ongoing school programs 
can and should be subject to accountability. Some innovative programs, however, 
by their very nature, cannot function within the same boundaries. Accounta- 
bility in research takes the form of integrity of purpose, procedures, and 
evaluation. Finally, any attempt to install strict accountability on a crash 
basis — as crash programs imposing other educational concepts have amply demon- 
strated — would not only be unwise but might guarantee failure of accountability’s 
primary purpose. 
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Workshop III 



Objective; To develop a paper which describes strategies for assessing educa- 
tional needs in the cognitive, affective, and psychomotor domains. 

RECOMMENDATIONS 

1. A needs assessment should be learner-oriented, that is, focused on the behav- 
ioral needs of children. 

2. A needs assessment should include the affective and psychomotor as well as 
the cognitive domain. 

3. State agencies should begin to address themselves to the task of developing 
competencies which will allow them to tackle the affective and psychomotor, 
as well as the cognitive domain. 

4. Local educational agencies should have both the prerogative and the respon- 
sibility for assessing their own needs and devising strategies to carry out 
the assessments. 

5. Planning a needs assessment should be preceded by determination of who should 
be involved in the planning, including personnel who will conduct the assess- 
ment studies and, at some point, representatives of all groups who will be 
users of the assessment results. 

6. Planning a needs assessment should include a determination of required re- 
sources, including advisory personnel, assessment costs, available strategies, 
and existing sources of data. 

7. A strategy for a needs assessment should include; 

a. development and/or selection of goals and objectives, 

b. design of instruments for data collection, 

c. determination of a means for precise analysis of data. 

8. A needs assessment she j.d be planned so that information will be available 
for decision makers as they need it. 

9. A needs assessment must include both reliability and validity checks of data 
collected. 

10. A needs assessment should include instrumentation appropriate for specific 

subpopulations, rather than the same administration for all respondents, re- 
gardless of background. 



11. A needs assessment should refer to a variety of data sources, with special 
attention to timeliness, accuracy, and manageability (accessibility and 
usefulness) of data. 

This report was developed around the nine topics on needs assessment 
provided by the U. S. Office of Education for discussion at the conferences. The 
ninv. topics explored were: 

1* nature and extent of planning 

2. who should be involved in planning 

3. what resources are necessary to conduct an assessment 

4. actual design of a strategy 

5. designing instruments for data collection 

6„ process of collecting information 

7. analysis of data 

8. determining reliability and validity of data 

9. translation of information into critical educational needs 

These topics are taken up, one by one, in the paragraphs which follow. 

The nature and extent of planning should encompass the other eight 
topics listed. The planning should include a formulation of the theoretical base 
for needs assessment and clear definitions of the objectives and the scope of 
assefssment . Emphasis should be placed on both short-range and long-range strat- 
egies. A realistic timetable should be built into the plan to provide for re- 
sults at the time they are needed for decision-making. And, finally, the plan 
should include the necessary logical linkage between assessment, evaluation, and 
dissemination. 

Returning to topic two, concerned with personnel involved in the plan- 
ning, the work groups felt, in general, that representatives of all groups of 
persons who were to be users of the needs assessment results and all persons 
whose specific skills would be required in conducting the needs assessment studies 
should be involved at one time or another in the planning. The latter groups 
should be involved, in particular, in a critical review and possible modification 
of final plans. These groups might include: 

a. appropriate representatives of State educational agencies, including 
persons responsible for administering State- and Federally-supported 
programs, 

b. appropriate representatives o^ u~OE, 

c. appropriate representatives of local educational agencies, 

d. representatives of professional organizations, 

e. appropriate consultants (e.g,, university-based evaluation specialists), 

f. representatives of school boards of education, 

g. representatives of State advisory councils for Title III., 

h. representatives of State boards of education, 

i. appropriate legislators and their staff consultants. 

It might be desirable to have a Needs Assessment Steering Committee to 
monitor, review, and revise strategies on a regular basis. 

Financial resources for use in planning and implementing a needs assess- 
ment might come from a variety of State and Federal sources. The possibility of 
a cooperative effort among State educational agencies in planning for needs 
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assessments, including a pooling of financial resources, should be explored. 

Other resources contributing to the planning and implementing of a 
needs assessment should include personnel from various agencies and special ad hoc 
groups which in the past had been especially concerned with needs assessments, 
existing strategies used in needs assessments, literature related to the subject, 
both technical and popular, and existing sources of data. 

Special competencies in system design, survey research design, statis- 
tics and measurement theory, sampling, and data processing will be necessary in 
che planning stages. Special competencies in developing behavioral objectives, 
survey research, statistics, sampling, and data processing will be necessary in 
conducting the assessment. Particular emphasis should be placed on having the 
data processing hardware and software necessary for timely treatment of the data. 

In considering the actual design of the strategy followed, it should 
be pointed out that needs assessment strategies typically have identified only 
deficiencies in student learning if, indeed, they were concerned with students 
at all. In planning foi: assessment, it is desirable to include the entire range 
of student achievement, i.e., various types of students in all areas cf learning 
at all levels of achievement. This would provide for student successes as well 
as student deficiencies. If particularly successful programs could be identified, 
the assessment strategy might provide alternatives to meet particular needs, in a 
diffusion/adoption process. 

A strategy for assessing educational needs should Include the follow- 
ing steps: 

a. The development and/or selection of student learning 
goals and objectives by the educational system . The 
objectives should be (1) behavioral; (2) measurable; 

(3) representative of cognitive, affective, and psy- 
chomotor domains; and (4) should include a time re- 
straint, that is, a specification of the length of 
time required to achieve a given objective or set of 
objectives. 

b. The collection of data about objectives . These data 

would fall into two categories: (1) Perceptions as 

to the relevance/iraportance of particular objectives 
as far as students — identified by grade, age, sex, 
ethnic group, and urban or rural residence--and con- 
sumers of the educational product- -grossly catego- 
rized as parents, former students, employers, col- 
leges, universities, and vocational and technical 
schools--are concerned; and (2) the criterion-re- 
ferenced test results of the extent to which objec- 
tives have already been achieved. 

c. The analysis of data . These data are analyzed and ranked 
according to their relevance/ importance and the extent 

to which they have been achieved* 

d. The use of data . The results are submitted to the 
educational policy-makers, who then determine what 
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actions arc to bo taken in respect to the identified needs. 

In designing instruments for data collection, consideration must be 
given to three different types: the statements of objectives, themselves; the 

criterion-referenced tests, designed to treasure the extent to which the objec- 
tives have already been achieved; and instruments designed to measure the per- 
ceptions of relevance and importance by different groupings of students and so- 
called consumers. Where it is appropriate to do so, data collection instruments 
should be field tested prior to actual use. 

The processes of collecting and analyzing data were covered by two 
brief statements. In connection with the first, specifications of actual pro- 
cedures used nust wait upon adoption of an overall strategy. In connection with 
the second, this is a highly technical function in statistics and data pro- 
cessing which, again, can be spelled out only after comprehensive plans have 
been adopted for the needs assessment. 

It may go without saying that both reliability and validity should be 
built into the strategy design for assessing needs. In this connection, the 
concept of validity may be the more difficult to pin down. Here, too, however, 
the rationale of identifying "consumers 11 may be apt. Persons who have had first- 
hand experience with specific products of a school system may be considered as 
"experts" in judging the product involved. Criterion-referenced tests, also, 
can contribute to validity in their function of determining what students already 
know. A third possibility exists in making comparisons between existing data 
and the results of the needs assessment. 

In turning to translation of information into critical educational 
needs, a distinction must be made between collecting information to be used in 
making decisions and in the decision-making process, itself. It may be that to 
the development of a strategy for making a needs assessment should be added a 
strategy for increasing the likelihood that the information will, in fact, be 
used. To determine which institutions and agencies are the potential users of 
the assessment results, it is desirable to determine which are responsible for 
meeting various needs. At the State i^vel, the State Department of Education is 
responsible for providing Statewide results to the State Board of Education and, 
perhaps, for recommending courses of action based on an interpretation of results. 

In summary, there was general agreement that a needs assessment should 
be learner-oriented and include the affective and psychomotor domains as well as 
the cognitive. Stating needs strictly in terms of institutional requirements, 
it was said, would be disastrous to the maintenance of public support for the 
educational enterprise. There was also general agreement on the direct relation- 
ship between learning objectives and statements of need. 

A needs assessment, it was agreed should be broader in perspective 
than Title III, alone, could make it. It should be instead a continuing ef'fox't, 
producing information for decision makers as needed. Needs assessments are 
essential steps in making any significant improvement in educational outcomes. 
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by Jams A. Hazlett 



I might say, to begin with, that the title of my speech was given to 
me, ,f Title III in Perspective. 11 

This is the third regional workshop of Title III coordinators , members 
of State advisory councils, and members of State department staffs. The first 
was held in Ft. Lauderdale, the second, in Philadelphia a week ago, and this, 
beginning tonight, is the third. Ray Warner, who has been introduced to you as 
the ranking officer from the U. S. Office of Education dealing with Title III, 
asked me if I would give the opening remarks at each of these conferences. I, 
unfortunately, was not able to go to Ft. Lauderdale and asked Helen Bain, a 
member of the Council, who is President-elect of the National Education Associ- 
ation, to act in my place. I guess she must have given a rip-roaring political 
speech — having just won the Presidency-elect of the NEA — and made it a pitch on 
behalf cf getting support for Title III appropriations in the Federal Congress. 
Anyway, when I went into the Sheraton Hotel in Philadelphia a week ago, someone 
said, "Gee, I thought we were going to hear that sweet little English teacher 
from Nashville again." Unfortunately, I accepted the invitation to come here and 
you won't have the opportunity to hear her, but I do bring greetings to you from 
the members of the National Advisory Council and want to share with you some of 
our thoughts concerning the administration and the past and the future of Title 
III. 



The Director of the Education Commission of the States often says that 
the Commission has a greater image than it really merits, and I suppose it sounds 
important to have a member of a national advisory council, presumably appointed 
by the President and overlooking the administration of a certain act, to come 
and talk about it. 

I would like to take just a minute to share with you the trials and 
tribulations of the National Advisory Council, if I might. It would deflate our 
image considerably, I'm sure. First of all, I would like to indicate who the 
members of the Council are: Mrs. Rosita Cota, a classroom teacher from Tucson, 

who, incidentally, is on a maternity leave from both a teaching position and the 
Council and who would like to have been here tonight. I know she sends greetings. 
She has been very much interested in Title III and an enthusiastic member of the 
Council. Helen Bain, I have already mentioned, another very enthusiastic person 
on behalf of innovation in education; the Honorable William Sanders, the State 
Commissioner of Education in Connecticut, a farsighted State official, has often 
declared that Title I V .I has made a tremendous impact on education in the State 
of Connecticut. Then we have Mayor J. C. "Pepe" Martin of Laredo, Texas, a 
layman who has come to most of our meetings; Mr. A. Louis Read, of WDSU, the NBC 
television station in New Orleans--Mr. Read has been quite interested in Title III 
Mario Fantini, a Program Officer for the Ford Foundation, the man oftentimes 




15 



behind the new o on Ocean Hill, Brownsville, those school communities in New York 
City which have made so many headlines; and Maynard Reynolds, Professor of Special 
Education at the University of Minnesota, a very thoughtful person, appointed to 
the Council because of his interest in Special Education, who has made great con- 
tributions • 



When we were appointed in 1968, there were twelve members on the Coun- 
cil, four for one year, four for two years, and four for three. The people whose 
terms expired the following January were Herbert Wey, at that time Associate Dean 
of Education at the University of Miami, Florida; Father Pierre DuMainc, who was 
the Superintendent of Schools of the Archdiocese of San Francisco; Bill Smith, 
head of the PACE organization in Cleveland; and Ruth Mancuso, former president 
of the National School Boards Association. We twelve people assumed our respon- 
sibilities as members of this Council, charged by the officials of the U. S. 

Office of Education, to do all that we could to cultivate within the educational 
systems of this country a search for creativity and innovation. We pointed out 
at the time that Title III, being one of the Titles under the Elementary and 
Secondary Education Act, was peculiarly designed to furnish one hundred percent 
of the money for the purpose of bringing out innovation in local school systems. 

We were charged with the responsibility of holding workshops, such as the one we 
had in Washington in October, over a year ago, bringing together Title III co- 
ordinators and Advisory Council members, helping them to see what Title III 
meant, helping them to design innovative programs, giving them some guidance and 
counsel on how to evaluate these programs and how to develop means for their 
dissemination. There was a great spirit of enthusiasm, really, at that time. 
Secretary Wilbur Cohen and Harold Howe were there and there was a great thrust, 

I believe, because of hopes of increased funding, which would have really brought 
money into the grass roots of our public school systems for the purpose of 
trying to do better the things that we were already doing. 

Well, by the end of the first year four members had retired; there 
have been no re-appointments yet; their vacancies have not been filled. We now 
constitute a group of eight people. I understand that all of the advisory commit- 
tees in the Office of Education, some twenty to twenty-five, and most of the ad- 
visory committees in the Department of HEW, someone has said two hundred or so, 
are being evaluated with respect to their roles and their membership. Now, this 
Council was the product of Congressional statute, was appointed by the President, 
and had four tasks: (1) to review the administration and general regulations of 

this Title, (2) to review, evaluate, and transmit to the Congress and the Presi- 
dent the reports submitted by the States, (3) to evaluate programs and projects 
carried out under this Title and to disseminate the results thereof, and, lastly, 
to make recommendations for the improvement of this Title and its administration 
and operation. 

We were provided a budget out of the S and E funds of the U. S. Office 
of Education, with which we were to hire an executive secretary and whatever 
staff was needed to carry out the mandates of this charge. We did hire Dr. 

Richard I. Miller, who was at the University of Kentucky, directing their Center 
for Educational Change and who has made quite a study of Title III projects 
during the life of this Title. Tho money, though, in the U. S. Office of Educa- 
tion is restricted, like that in other agencies, and we have not been able to 
find the sums of money to do the kinds of things that we thought we originally 
would do--iaaking visitations to local projects, hiring consultants to assist 
in the evaluation of projects, identifying those things which seem most promising, 
and then disseminating information about them to not only the Congress for its 



annual report but to the country, at large. We have not been able to do these 
kinds of things. And, because of our Council's depletion in rank and because of 
the rather glootny financial picture that is true of Washington today, I suspect 
that some of the things I will say tonight may not prove to be too inspiring or 
too stimulating, I think, nevertheless, we must face the situation very frankly. 
Even if Title III loses its original thrust, even If it is not financed up to the 
,’vcls first authorized (and certainly that's not going to be possible), and even 
if it tends to lose some of its uniqueness, I think the past history of this 
Title has produced some by-products--sone things which the U. S. Office and the 
States have had experience with--which, if they can be infrsed into your own re- 
spective plans, in your own States, will surely demonstrate that Title III has 
been a valuable contribution to American education. 

Now these people whom I have named on ths Council, for the most part, 
are not intimately connected with education. Some of them are laymen, like 
"Pepe" Martin of Laredo and Mr. Read of New Orleans, and all of us have had to 
rely for our two reports on material and information and analyses of Ray Warner 
and his people in Washington and of our own staff at the University of Kentucky. 

So we, as a group, are really only able to make certain general observations 
about the operation of Title 111. 

Before 1 list these particular observations, I'd like to take just a 
minute, since the subject of my talk is "Title III in Perspective," to remind 
you that Title III was but one part of a larger package, the Elementary and 
Secondary Education Act. This act was designed previous to 1965 and finally funded 
in that year on the premise that the so-called establishment of the public school 
system — the superintendents of schools, like me in Kansas City, and the State 
cocmissioners of education (of which you cannot be guilty, Dr. Shofstall, since 
you at that time were not one) — was really incapable of making the transformation 
which was necessary in education. At this point, our society was suddoi ly awakened 
to some things we as educators already knew: that there were large segments of 

our population which were really not achieving in terms of their abilities or in 
terms of the social demands which would be made on them. This awakening came at 
a time when the Civil Rights question was becoming important and when there was 
a move to eliminate poverty from our midst. So something had to be done. Those 
who were taking the leadership in education at the Federal level, somewhat sus- 
picious of the capabilities of the existing school system, declared themselves 
to be junior partners in education and designed a package which thrust money at 
the disadvantaged, in Title I, and at innovation, in Title III and IV, and then 
said to the State departments in Title V, "now here's soma more money to build 
staffs; let's oee if you can't do something with it." One ranking official at 
the time said that if, with the help of these funds, there is no improvement, 
the junior partner migAt very well become more of a senior partner. That was 
the existing mood, Z think, four or five years ago. 

Then, in 1960, there were some revisions in the Act and these revisions, 
at least as far as Title III was concerned, turned the responsibility for the 
administration of this very innovative Title over to State departments of educa- 
tion. In the first year of a two-year period, seventy-five percent of the funds 
were to be administered by the States; in the second year, all of the funds. 

There wes a shift, then, in the so-called balance of junior and senior partner- 
ship; and the suspicions were voiced that the States were not capable, were not 
able to tool up, and did not have the manpower to provide the same kind of 
leadership which would de”elop creativity in new projects. There was also some 
suspicion that many States would use this money as a kind of a general grant and 
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"divvy it out" on some kind of basis. Examination by our staff during this year 
of transition shows that this has not been true, that the State advisory councils 
and the Title III coordinators, by and large, have accepted the responsibility 
of maintaining the focus for the moneys on innovation and on creativity. We 
have been pleased with the conscientiousness ,*ith which Title III coordinators 
and, by and large, the State advisory councils have assumed their roles. They 
have analyzed State plans and reports from the States and these will be summarized 
in the report which will be released on the 20th of January under the title of 
The Rocky Road to Innovation . We sense to some extent the loss of interest in 
the U. S. Office of Education with respect to Title III, not meaning Ray Warner 

and his people at all, but meaning simply the feeling of the total administration 

towards the focus of this particular title. 

When I was c member of the old Office of Education Advisory Committee, 
under Harold Howe and Nolan Estes, there seemed to be a great interest in using 
Title III as a transforming agent. That mood is rot there now and in part it is 
quite natural because the responsibility has gone to the States. But one of our 
reconsnendations and certainly our hope is that the same mood which enveloped 
Title III at its inception will continue at State and local levels. 

Now, we are in a period, as you all know, of financial crises in so far 

as the funding of education at the Federal level is concerned. The President has 

taken a clear stand that there will be no increase in expenditures, that the num- 
ber one problem which we must face is to cut down on inflation, and one way to 
do this is not to increase government spending. We recognize the importance of 
this objective. But when we hav^ been gathering momentum in just two, three, 
four, five years in developing programs which we think are meaningful, then we 
must pause and wonder to some extent if our efforts have not been in vain or if 
there is not some way in which we can keep the mom v ntum rolling. 

Now, just some general observations with xespect to the Council's feel 
for Title III. The Council noted that its name is the Council on Supplementary 
Centers and Services. We Teel that perhaps many of the members of Congress and 
others believe that Title III was established to provide Supplementary Centers 
ai:d Services only--to give money to school districts which did not have psycho- 
logical services or libraries or science centers or the like so that the districts 
mighr. provide these added services. We of the Council, and I think the U. S. 
Office of Education and those in the profession, have always felt, however, that 
the real thrust was not to create supplementary centers and services--which should 
be provided out of State and local funds — but to use Title III for the designing 
of programs and projects which are unique and distinctively new, in so far as 
pojsible. When, under HR 514, Title II and III of ESEA and, I believe, Titles 
III and V of NDEA were lumped together, I analyzed that lumping as being a way 
to pool things and services, like Title II did with library books and Title III 
NDEA with science equipment. Put things and services together and, if that be 
the thinking, the whole focus, the whole import of Title III towards fostering 
distinctively different kinds of programs, has been lost. But you know, and the 
U. S. Office of Education has continuously advised, that supplementary centers 
and services is not the real import; and yet I feel that that is a very important 
consideration In this whole matter. Perhaps we failed in not being aware of it 
and in not acquainting Congress with it. Naturally, too, we are concerned about 
the funding level. This year's funding, rather 1969' s funding, was slightly 
below 1968 funding. The President's budget is calling for $116,000,000 compared 
to $165,000,000 for the year just ended. Because of inflation, just to maintain 
present »vels takes at least an eight percent to ten percent increase. And so 
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we are confronted with this very real situation of restricted amounts of money. 
Many of your projects, however, are terminating. There may be some new funds by 
which you can invest in new projects. Of course, the timing of the appropria- 
tions has been a problem for a long time and it is worse now than it has ever 

been, with no appropriations for educational purposes at all. The President, as 

you know, has indicated that if Congress does not pass the appropriations bill 
he will reconvene it on the 26th of December. Maybe that will have some effect. 

The Council, in addition to stating a belief in the need for increased 
funding, also concludes that there should be no pooling of Title ITi with any 

other Titles, that it should stand alone, that it is unique and different. Now, 

out of the examinations which we have made of hundreds and hundreds of projects 
and out of testimony and letters and printed materials we have received, we know 
that Title III has made an impact on millions of children and thousands of 
teachers. This is a story that more and more needs to be told and yet our Council 
was charged only with the dissemination of evaluation of projects. When we got 
into publication of PACEreport, with which most of you are familiar, we were 
criticized because we published articles dealing with how to effect change, how 
to evaluate, and how to carry on dissemination programs-- topics that went far 
beyond the evaluation of specific projects. But we know the impact has been 
great and that it has made a change in the lives of many people. Out of Title 
III has come some by-products which, even if Title III should continue to shrink, 
are the kinds of things I would hope all of us would continue to recognize as 
being of value. One such by-product was the utilization of lay people in planning 
and evaluating. The State advisory councils are amenable to that, bringing to- 
gether men and women outside the profession, from different walks of life, and 
from minority groups, and giving them a chance to visit projects and to make 
evaluations . 

We are living in an age, as you all know, when community action is a 
kind of by-word- -community action in 0E0, community action in Model Cities, 
local boards of education deciding major problems once entrusted to elected 
officials. This is a technique by which lay participation can be utilized ac 
the State level. It is something I hope would continue and be used further. 
Another valuable by-product is the requirement for needs assessment--to take a 
good, honest look at the educational strengths and deficiencies in any given 
State, to note that there are variations in needs in different geographic areas, 
and then to begin to think how programs can be developed to meet these deficien- 
cies and needs. If one reflects upon this long enough, he will begin to see 
that this analysis is beginning to take us away from the old idea that education 
is a ready-made product, which is handed to everyone regardless of who he is or 
where he lives or what hip background is, and that it instead leads us to deal 
directly with specific problems under highly varied conditions . 

The emphasis on evaluation- -very important- -upon the spelling out of 
objectives in behavioral terms and then upon evaluating these objectives, is 
another important offshoot of Title III. Now, our Council has had some diffe?r- 
ences of opinion as to whether you can really spell out all these objectives in 
behavioral terms, but, at least, this is certainly a step in the right direction. 
We have gotten down in the last year or two, when we consider innovation and 
change, to the idea that change for change's sake is no good. Many of you 
probably said you never thought that anyway, bu. I have heard distinguised 
professors of education stand on public platforms and say that change for change's 
sake is important — it doesn't make any difference about substance, if you can 
modify the traditional ways of doing things it gives invigoration to the people 



who are involved. It focuses their energies; it creates a Hawthorne effect; 
it's good--just change, itself. But now we have heard, in the last year or two, 
more and more about change for which you have to account. Accountability is be- 
coming one of the popular educational concepts of the day and one which I think 
we really need to bear in mind. 

Well, it’s the Council’s wish, and hopefully it’s your wish th\t Title 
III will maintain itself as an independent educational program, adequately funded 
and free from being a part of any other program. Whatever influence at this late 
date anyone of you has in influencing the Senate Appropriations Committee, or 
any other groups which have the power to fund, should be exerted. We must con- 
tinue to push for increased funding aad for the focus of effort and money upon 
continuous innovation at the local school level. Yet, if there is a re-examina- 
tion of the evolving patchwork of Federal aid (and when you look at the history 
of Federal aid to education, it is a kind of crazy quilt), and if we can ever 
get a comprehensive definition of what Federal support should be, we can always 
remember that Title III stood as a symbol — a symbol that there must always be 
responsible innovation to keep pace with changing times and changing needs. In 
addition, you, in your particular roles, can infuse into all your activities the 
things that Title III stood for — needs assessment, lay participation, cooperative 
endeavor, rigid evaluation. If this can be done through the experiences we have 
had with Title III, even if it is not funded where we would like it to be, it 
will have made a real contribution, I think, to American education. 
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by Leon Lessinger 



The American education system today is experiencing the most sustained, 
diverse, widespread, aud persistent challenge ever to confront it. Virtually 
everyone agrees that something has gone wrong, that corrective action is needed. 
Congress and State legislatures have responded to this crisis of public concern 
by providing additional funds, but are increasingly dismayed that puzzling edu- 
cational problems persist. 

In principle, the American educational commitment has been that every 
child should have an adequate education. This commitment has been stated in 
terms of resources such as teachers, books, space, and equipment. When a child 
has failed to learn, school personnel have assigned him a label--"slow," or "un- 
motivated," or "retarded." Our schools must assume a revised commitment-- that 
every child shall learn. Such a commitment includes the willingness to change a 
system which does not work, and find one which does; to seek causes of failure 
in the system and its personnel instead of focusing solely on students; in short, 
to hold the school accountable for results in terms of student learning rather 
than solely in the use of resources. 

If schools are to be accountable for results, a new approach to the 
basic mission of the schools is necessary. In the first place the focus must 
shift from teaching to learning. Second, the schools will cease to merit credit 
solely for their ability to screen and sort in a rutted roadbed toward college 
or the discard pile. Third, a technology of instruction based on specific 
learning objectives will start to build. Finally, a rational relationship may 
be established between costs and benefits. 

Often in science and in other spheres of thought it is not possible to 
confront an important idea directly. For example, the phenomenon of electricity 
is understood by its effects — flow, resistence, pressure. It may be helpful to 
approach accountability in a similar indirect manner. The ideas of audit, per- 
formance contracts, developmental capital, and educational escrow accounts con- 
stitute the rather basic, primitive attributes of the concept of accountability 
for learning results. 

I. Performance Contracts To Achieve Ac countab il i ty in Education 

An experimental dropout prevention program illustrates an approach to 
achieving accountability through performance contracts with private enterprise. 

A description of this project and some generalizations about its possible impli- 
cations can illustrate the use of performance contracts to achieve accountability 
in education with components other than or including private enterprise. 

The Texarkana, Arkansas, School District was granted funds to conduct a 
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dropout prevention program under Title VIII of the Elementary and Secondary Edu- 
cation Act. An "actor's agent" was employed in the form of outside technical 
assistance from a nonprofit firm to help them translate their compensatory train- 
ing objectives, e.g., reading, arithmetic, study skills for disadvantaged stu- 
dents in six junior and senior high schools, into performance criteria for com- 
petitive bid by private contractors. The firm also helped them to develop the 
Request for Performance (the document against which the bidding took place), to 
oversee the bidding process, to develop the actual performance contract with the 
successful bidder, to communicate with school staff and community, and a host of 
related activities. Dorsett, Inc. subsequently was awarded a performance contract 
built around the following provisions: 

1. Students who are two or more grade levels below standard in basic 
skills and whose family income is $2,000 per year or less are to 
receive a training program for up to three hours per day in a por- 
tion of the school plant. The students remain in the total school 
program to receive other school benefits. 

2. The contractor agrees to be paid only on the basis of a stipulated 
amount of money for each student who successfully completes the 
training program. 

3. A penalty is assessed for those student? who do not achieve speci- 
fied performance levels . 

4. Six months after the termination of the project, school officials 
have a right to re-assess student performance. If it is less than 
the specified level achieved, a penalty may be assessed. 

5* The school system, not the contractor, selects the students* 

6. The training program of the successful bidder must be cost-effective 
and not labor intensive. 

7. The contractor agrees to train school personnel so that the school 
system can carry on successful practice after the project is ter- 
minated* 

The assumption behind the Texarkana program is that a private contractor 
in concert with regular school personnel in the overall school setting, performing 
both an institution building and a direct instruction service, will have greater 
freedom to innovate and may be more successful in motivating students than the 
regular school system has been. 

Performance contracts are not new to education. But the concept of 
holding an educational agency accountable for results is. When a student is 
able to demonstrate in concrete terms what he has or has not learned, educators 
will be in a better position to judge where and why a program succeeds or fails 
and make the necessary changes to achieve success. 

In the main, educators have failed to develop performance criteria for ’ 
measuring the effectiveness of instructional programs, and many programs are now^ 
underway which at no point describe what students are expected to gain from their 
educational experiences. 
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Instead of vague promises to provide students with "an opportunity to 
learn to communicate effectively/ 1 instructional program objectives should be 
stated in terms as specific as these in the following example: 

Given three days and the resources of the library, the student com- 
pleting this program will be able to write a 300- to 500-word set of 

specifications for constructing a model airplane that another student 

could follow and build. 

There are and should be larger objectives in education that are diffi- 
cult to define and impossible to measure as the consequence of any given program. 
The training components of education, illustrated in the basic skills of reading, 
arithmetic, and the like are most amenable to performance contracts. 

But the fact that many results of education are subjective and not 
subject to audit should not deter us from dealing precisely with those aspects 
of education that lend themselves to precise definition and assessment. Rather, 
it demands that we do make maximum use of these individual parts that tell us 
what the change in the whole has been. 

II. Independent Accomplishment Audit 

Independent fiscal reports have been applied to the fiscal side of ed- 
ucation for many years with great success. The independent fiscal audit not only 
has resulted in the virtual abolition of shady financial practice, but the pro- 
fessional recording, classifying, and interpretation of the economic facts of 
the enterprise have been done in a manner designed to produce data which encour- 
ages and permits effective management. 

The Independent Accomplishment Audit (IAA) relies upon outside judgment 
of results or accomplishments. It has six essential parts: the pre-audit; the 

translation of local goals into demonstrable data; the adoption/creation of in- 
strumentation and methodology; the establishment of a review calendar; the as- 
sessment process; and the public report. 

Each word in the expression "independent accomplishment audit" was 
chosen to express the concept. A close look at each of the elements may be a 
useful way to display its meaning. 

Independent : 

The community served by a school usually has little choice in the in- 
formation and reports given it by the teachers, administrators, and board of 
trustees . 



A distinctive characteristic of the IAA is the concept of a third par- 
ty review to assess the "truth" as seen by the independent reviewers without 
sentimental, defensive, protectionist, or financial influence. Outside objec- 
tivity can nurture respect for the re ort as an honest accounting of what has 
happened to children's attitudes, skills, and the level of knowledge in relation 
to locally established objectives and goals. 

A ccomplishment : 

Results are the products, services, or other effects created by the 
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school. Results achieved stand in contrast to resources consumed by the school. 
Every organization has, or at least is intended to have, outputs even though 
they may not be readily measurable or even clearly definable. In other words, 
every organization does something and that something is its output. Employee 
development, satisfaction, and morale are relevant outputs as are taxpayer satis- 
faction and support, but student learning is the primary concern--the raison 
d 1 etre of education. Student accomplishment is the prime output to be judged by 
the IAA. 



Judgments can be made on the basis of interviews, observations, and 
instruments such as tests or video-tapes. Through the use of small-sample sta- 
tistical techniques, judgments of small numbers of students can indicate the 
performance of ^he entire student body served. The range of judgments can be 
specified in a classification ranging from crude to rather exact. This full 
range u£ assessment can be utilized in the IM. 

Audit : 

An audit is a standard review conducted by someone whe is qualified 
and trusted to make objective reports. It is anticipated that auditors will 
come from the ranks of professional educators, the laity, universities, and pri- 
vate enterprise. Training will be essential. 

The following description of the IAA process itself should further an 
understanding of its relevance as a stimulus of accountability. 

1. The auditor selected by the school system starts the IAA process 
by discussing with the staff (students and community can be involved at this 
stage) the objectives and plans of the particular program to be reviewed. 

2. In concert with local people, the auditor determines a clear for- 
mulation of the evidence that will be used by the local people to indicate that 
the objectives have been met and the conditions that will be used to elicit the 
evidence. 



3. The auditor determines the instruments, such as tests, question 
naires, standardized interview, and the like which will be developed or secured 
to gather the evidence agreed upon. 

4. An agreement is secured in writing which indicates the nature of 
the reviews, where they will be held, how long they will take, when they will 
occur, and who is responsible for arrangements, the nature of the arrangements, 
and other logistical considerations. 

5. The auditor carries out the procedures agreed upon as codified in 
the review calendar. 

6. The auditor files a report at an open meeting giving commendations 
and recommendations as they relate to the local objectives. The report is de- 
signed to indicate in specific terms both accomplishments and ways the program 
may be made more effective and/or efficient. 

The IAA is a new technique designed to put local school personnel and 
the clients they serve in a problem-solving mode of thinking. It is built a- 
round a financial core since money is a common denominator for the heterogeneous 



elements of inputs but its focus is upon student attitudes, skills, and knowledge . 
Out of the IAA a whole range of useful by-products may be anticipated. First, 
it may lead to a knowledge of optimum relationships between outputs and inputs, 
i.Co, those that ?re cost-effective or otherwise to be values. 

Second, it can form a basis for the discovery and improvement of good 
practice in education. 

Finally, it can renew credibility in the educational process, effect 
more responsiveness as to the needs of children, and supply the understanding 
necessary to produce change. 

III. Developmental Capital and Accountability 

Money available in a predictable and secure manner for responsible in- 
vestment to produce results is the energy of accountability. Developmental capi- 
tal is the money set aside for investment by school personnel in activities which 
produce the results described in the IAA. Business typically budgets amounts 
varying from three to fifteen percent for research and development designed to 
produce better products, better service, more sales, or more capability to pro- 
duce these items. Until the passage of the Vocational Education Act of 1963 and 
the Elementary and Secondary Education Act of 1965 there was virtually no money 
earmarked for this general purpose in education. With the passage of these Acts, 
there is now approximately one -third of one percent available for this type of 
investment. Perhaps the best known of these programs involving "risk" capital 
is the highly successful Title III program of innovative and exemplary projects. 

The basic purpose of developmental capital is to provide a financial 
resource to stimulate and sustain re-examination and modernization of the educa- 
tional system. The investment of "risk" capital can generate new educational 
traditions by applying the developmental aspects of business success to the pub- 
lic sector. 



School systems too often are characterized by archaic budgeting sys- 
tems; poor use of buildings, staff, and equipment; low salaries, unrelated to 
performance; inadequate personnel development; poorly developed promotion sys- 
tems for teachers; outmoded organization; inadequate equipment and materials; 
primitive technology; and repetitious and uninspired instruction. 

Effecting necessary change requires discretionary funds which are not 
now available to local school leaders. In the absence of an infusion of new 
monies for development, dissemination, and installation of new products and prac- 
tices, the gap between demand for higher quality education and performance is 
likely to widen further. Title III ESEA has been and is a dynamic and vital 
source of such changes --albeit on entirely too small a basis. 

With development capital set-asides, renewal can be directed through 
Federal, State, and local channels, and activity can be aimed at improving man- 
agement leadership capabilities. All three sectors of Government can work in 
conjunction with one another and with the private sector. At each level a par- 
ticular focus can be attempted. For instance, Title III ESEA funds at the 
Federal level can be applied to "high risk" investments and identification of 
successful practices around the nation. State educational leaders need capital 
to increase the effectiveness of their State departments of education and to 
furnish incentives to reward performance on State priorities. 



There are many ways In which developmental capital, such as Title III, 
might be used at the local level. One possibility, called producer-consumer 
school district ccntract development, involves the sharing of good, locally de- 
veloped educational practices. Pieces and parts of exemplary models exist 
throughout the nation. But missing is an effective way to insure adoption, dif- 
fusion, and dissemination. Title III would target directly on these sorts of 
problems and, in addition, the producer-consumer contract development program 
would provide money for local investments to atudy, plan, train personnel, and 
install good practice which the local people want to obtain. 

An example of the use of developmental capital in a local school dis- 
trict may serve to illustrate the concept and its relationship to accountability 
for results. The superintendent of a large California school district was al- 
lowed to manage approximately one percent of the operating budget of the school 
district, some $250,000, as an investment account. Board policy also permitted 
the set-aside of one percent of all funds raised from Federal, State, and pri- 
vate sources. For the period 1965-68, with the assistance of an elected teachers* 
group called the Academy of Instruction and the cooperation of students, admin- 
istrators, community members, and the board of education, this set-aside account 
was used to invest in competitive teacher/s tudent/administrat' r proposals tied 
to demonstrable objectives. 

These innovative and exemplary activities have brought beneficial 
changes in student accomplishment , teacher morale and effectiveness, and admin- 
istrator behavior. Such things as a Know and Care Educational Resources Center, 
a Zero Reject Reading Laboratory, a Physical Fitness Testing Center, a Humani- 
ties Center, and incorporation of vocational programs into the fundamental re- 
organization of an entire school are a few of the results. The one percent set- 
aside was used as a "rudder 11 to cause change affecting the entire budget. 

IV. An Escrow Account in Education 

Accountability may be enhanced in public education by the introduction 
of competition from either inside or outside a school system. This competition 
might be the product of fiscal incentives and a utilization of the market mech- 
anism. For example, one way to obtain competition in compensatory training for 
disadvantaged students would be the creation of an escrow account. From this 
account, designated parents in the poverty category could obtain a voucher for 
use in accredited schools or private agencies according to performance stipula- 
tions. A plan is outlined below. 

1. A school system, receiving Title I dollars, sets aside an amount 
in an escrow fund. Vouchers, equaling the amount of the escrow fund--the value 
of which is determined by the child* s need for compensatory education--are dis- 
tributed to parents of a selected number of children. 

2. Parents of the selected children choose the educational agency to 
which they want to send their child for a portion of the day for compensatory 
training on a guaranteed performance basis. 

3. The school systems and other alternative education service suppliers 
negotiate individual contracts with the parents, guaranteeing a specific level 

of achievement required for payment to the voucher. 

4. Through this negotiation, the parents and the schools obtain 
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incentive leverage to give the children the truest and surest form of compensa- 
tory education. 

Such a program might have several major effects: 

!• It can create a true educational market mechanism in which all in- 
terested suppliers of educational services compete to please the consumer. Sup- 
pliers may be: 

a. Public school systems and individual schools within the sys- 
tem. 

b. Private schools. 

c. Private corporations. 

2. It can increase parent and community participation in educational 

matters. 

3. It will foster accountability. Whatever the contract is--to raise 
the goals of the student, or to raise his reading level--the supplier will be 
held accountable for the success of his services. 

4. It will increase the overall quality of instruction because the 
consumer will pick the better schools through the spur of competition. 

5. It will make education relevant to the desires of the parent and 
the needs of the child. Lack of relevance is one of the major criticisms of 
education today* 

Conclusion 



Discontent with the schools as they are is widespread. Much of this 
discontent arises because so many of the students in the central city schools 
fall two or more years behind the national average in reading and arithmetic; 
much of the discontent arises because so many children dislike school and drop 
out; much of the discontent arises because many students and educators find the 
schools rigid ? and more concerned (some of the schools say necessarily concerned) 
with discipline than with education. Our task then is, first, to utilize what 
we know and what we have learned through the use of these minuscule risk funds 
available during the past few years and to focus increased discretionary funds 
on these "Winters of Discontent Title III is a proven vehicle. 

Our task then should be obvious. 
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by B. Aldcn Lillywhite 



It's a pleasure for me to be here to talk to those who are responsible 
for establishing policies in ESEA Title III programs as well as those who admin- 
ister them. Policymakers, I find, have a different perspective on matters affect- 
ing the schools than do administrator s--and I find the difference very interesting 
and inf ormat ive • 

Despite these differences in perspective, I believe both the policymakers 
and administrators would agree that the American education enterprise is in great 
trouble. I believe they recognize the present turmoil in education and they may 
even agree that much of the educational establishment is presently held in low 
public esteem, if not outright ill repute. 

There are a substantial number of informed people today who think that 
for all practical purposes the educational enterprise has made little progress in 
the past fifty years. Some would say, with considerable justification, that the 
state of the art in many local educational agencies resembles a 17th century 
cottage industry, with little likelihood of any significant change in the near 
future. I do not know how many of you share this dreary view. It certainly is 
obvious, however, that education has failed for a substantial portion of the 
children it is supposed to serve. 

Many people wonder why this is so. They wonder because, in spite of the 
poor public image, many promising and productive things are emerging today in our 
educational practices. I won't enumerate them because they are well-known io 
most of you. Yet, a feeling of failure pervades much of the public discussion 
about education and, I think, justifiably so. Some people even wonder whether 
public education can make the adjustments necessary for the required results or 
whether some other more responsive organizational arrangement will have to be 
developed. 



As I worked over this speech last night I wondered why we have come to 
our present state of discontent with public education; I decided that we should 
look back for a moment at what has been happening, to see if we could find some 
answers to these puzzling questions and some suggestions for future action. Some 
of you here may disagree with concepts presented in this speech; if so, we should 
have a livelier discussion. My principal purpose is to stimulate thinking and 
discussion which, I hope, will lead to appropriate future action, and to indicate 
the role of Title III in this change process. 

Fifty years ago, even twenty-five years ago, people could get along 
reasonably well regardless of the kind or quality of the educational program 
they were receiving. If a youngster was failing in school, he could drop out and 
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still get some kind of job. Education--particularly higher education, but in- 
cluding secondary, as wcll--was still a frill, frosting on the cake. And although 
it was pleasant to be 3blc to quote Shakespeare, to manipulate the binomial theo- 
rem, and to say "All Gaul is d ivlued into three parts'* in Latin, as long as 
you could read, write, and figure, you could make a living. Employment require- 
ments began inching up after World War I, but any deficiencies in the schools 
that might otherwise have been revealed through the inability of uneducated 
Americans to get or hold jobs were hidden first by the depression behind a break- 
down in our economy and then by World War II in a need for manpower of every 
description, regardless of its quality. 

The result was that public education had mostly satisfied customers and 
the dissatisfied tended to leave without complaining. The children of the afflu- 
ent sought educational improvement in private schools and children of the poor 
ignored educational failure by seeking employment. Like some public utilities, 
public education had a captive market; it was a benign monopoly, and--again like 
public utilities--its occasional critics were dismissed as cranks. 

But now things are different. Requirements of the past do not meet 
present day needs and education is in the midst of change. It has been forced 
out of its protected environment by such major social Issues as: 

— more demanding job requirements stemming from the increased intellectual and 
technological complexity of modern employment; 

— a new militance among ethnic minorities proceeding from the tension between 
extraordinary affluence among some Americans and continuing poverty for others; 

— a new perception of social injustice arising from a communications explosion 
which, through television, brought the squalor of Harlem into the living rooms 
of Scarsdale and the comfort of the suburbs to the tenements of the ghetto. 

Indeed, it seems we are living in a time when the most stable thing in 
our society is change. As the Angel Gabriel of Green Pastures described it, 
Everything nailed down is coming loose. 11 The changes taking place are abrasive. 

To those who want to stand fast on **the way it's 1 spozed to be** they are fright- 
ening. They threaten the securi^’ of those who view education as the sole province 
of the professional educator acting with the concurrence of a policy-setting board. 
The changes are confusing and often misinterpreted by those who know that change 
is essential to achieve desired goals but do not know exactly what it should be. 

One of the effects of social change, as we have seen in the field of 
civil rights, is that it generates hope and even greater concern for more progress. 

We all are able to recognize that there i3 considerable questioning and 
resentment on the part of many of today* s parents as they watch their children 
fail to obtain the education necessary in an increasingly complex society. The 
parents of those leaving school, those marked failures, and the in-school "dropouts" 
say with some justification, "Ify kid is not dumb. Why can*t you teach him to read 
or do math? 11 

Small wonder that news of computer technology and outdoor classrooms 
fails to impress communities where 

- as many as seventy percent of the students drop out before graduation, 
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- many read at levels three or more years behind national norms, 

- few can compose an essay in acceptable English, and 

- few can perform simple arithmetic operations accurately. 

The . - arc among the very problems that Title III was designed to solve. 

In principle, the American educational commitment has been that every 
child should have an adequate education. This commitment has been stated in 
terms of resources such as teachers, books, space, and equipment. When a child 
has failed to learn, school personnel have labeled him "slow," or M unmotivated ," 
or "retarded." 

Our schools must assume a revised commitment: that every child shall 

learn . Such a commitment includes the willingness to change a system which does 
not work and find one which does, to seek causes of failure in the system and its 
personnel instead of focusing solely on students, a willingness, in short, to hold 
the schools accountable in terms of student learning, rather than in terms of the 
use of resources. 

Title III has brought change, but it is clear that there will have to 
be many more changes--in attitudes, methods, processes, and goals--before our 
schools can fully meet the demands placed on them today. 

Our public elementary and secondary schools enroll 44 million students, 
employ 1.9 million teachers, and account for the expenditure of $30 billion in 
tax funds annually. We have all kinds of statistics on where the money goes. 

We can pin down per capita expenditures in any school district in the country, 
state how much any of them are spending for construction and service on the debt, 
and enumerate pupil- teacher ratios until the sun goes down. 

All this seems to add up to learning more and more about less and less 
that is really relevant to the education of our children. 

But we have virtually no measurement of the results that our enterprise 
yields. We do not know, for example, what it costs on the average to increase a 
youngster* s reading ability by one year; all we know is what it costs to keep 
him seated for one year. All the indices we have measure our skill as financial 
managers; not a single one evaluates our effectiveness as educational managers. 

It would make much more sense if — as one expert has proposed--we moved from a 
"per pupil cost** to a "learning-unit cost, 11 to focusing on the level of learning, 
not the maintenance of children in school. The definition of effective teaching 
may well be measured in terms of how well the students learn. 

There have been a few moves to measure the results, however, and ample 
proof of failures has coroe out despite the schools* reticence. Today, about one 
of every four American children drops out of school somewhere between fifth 
grade and high school graduation. The Army Surgeon General reports that, in 1965, 
one of every four eighteen-year-old males failed the mental test for induction 
into the service. And hundreds of thousands of parents, particularly of minority 
children, reacting to this information, contend that their children are not 
stupid--but that perhaps some educators are incompetent, or the methods they use 
are inade qua te • 
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Why did these proofs of failure have to come from outside the schools 
rather than from within? Why haven't we been alert to the symptoms of failure, 
so that we could learn tc cure it? If one airplane in every four crashed, as 
passengers we would be in a lynching mood. If one automobile in every four went 
out of control and produced a fatal accident, Detroit would be closed down to- 
morrow. Yet our schools --which are much more important than airplanes or auto- 
mobiles — are somehow failing one youngster in four and we have not acted effective- 
ly to arrest the social and economic fatalities which every school dropout repre- 
sents. It is no wonder, then, that critics, such as Paul Goodman and Edgar 
Friedenberg, feel that many dropouts are better off on the street than in the 
classroom. And it is no vender that some parents--! think of those who took over 
I.S. 201 in Harlem, for example — feel that rank amateurs can do no worse job of 
running a school than the self-styled pros. In refusing to give our client: 
proof of performance, while they gathered their own proof of our flops, we have 
set the stage for the raucous and bitter confrontation with power-blocs common 
to the American educational scene today. Moreover, we have dissipated that con- 
fidence which the public once reposed in its educators, and we have almost com- 
pletely eroded our own claim to professionalism. 

If we are to assert that professionalism, if we are to meet the demands 
for change, we must join accountability to public education* And by "accounta- 
bility" I mean much more than such simple indices as numbers of dropouts and the 
results of reading tests. I mean the factors which produce specific educational 
results with different groups of children, through investment of specific amounts 
of financial resources. 

What is implied here, obviously, is a much more sophisticated kind of 
accountability than ninety-nine percent of us have been accustomed to. We have 
been traditionally concerned with quantitative input, with the amount of financial 
and human resources that have gone into buildings, into teacher salaries, into 
textbooks, and so forth. Listing expenditures by such categories is a legitimate 
way of showing the public where its money has gone. 

But it has nothing to do with showing the public the results of its 
investment in education. It is not a legitimate way of demonstrating qualitative 
output, c r what I call "accountability for results." The distinction is easy to 
make as soon as we ask what results the public wants for its investment. 

Does the public want, as a result of having spent X millions of dollars, 
the service of Y number of teachers? Does it want to own a given number of text- 
books, of test tubes and analytical balances, of trombones, and world globes? 

Obviously not. What it wants from its investment is educated children, 
able to meet their own needs and society's needs to the full measure of their 
potential. 



One problem is that we educators tend to state our purposes in vague 
terms. "To create responsible citizens" is one of our noble objectives. "To 
develop an appreciation for good literature or the excitement of science and 
mathematics" is a further example. Such generalized, global objectives are grand 
and beautiful — and totally useless to the teacher charged with fulfilling them. 

What the teacher needs is something a little less grand and considerably 
more specific. She needs something such as: 



O 
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"Each child who completes this course of study should be able 
to read two hundred words per minute with ninety percent com- 
prehension, 11 

Or, "Each child who takes one month of this physical training 
should be able to demonstrate the proper form for three indi- 
vidual sports activities: archery, pole vaulting, and the 

standing broad jump," 

These statements are precise in terms of the learning results to be 
achieved. Any teacher, parent, or student can understand them. 

They don't tell us how the stated objectives can be achieved, but they 
tell us specifically where we are going. 

In the dropout prevention program, the bilingual education program, and 
in other demonstration activities administered by the Bureau of Elementary and 
Secondary Education, a number of approaches are being tried in an effort to test 
the notion that knowing where we are going will improve our chances of getting 
there. 



These efforts to account for school actions in terms of student learning 
are what I mean by "accountability." The schools normally account for their 
actions in terms of the purchase of various resources and school needs. 

One approach being tested along these lines is the "performance con- 
tract," On this approach, anticipated student learning is stated in advance and 
incentives are offered to those able to produce greater results than specified. 

The dropout prevention program in Texarkana, Arkansas, illustrates an 
approach to achieving accountability through performance contracts with private 
enterprise. The local school district translated its training objectives in 
reading, arithmetic, and study skills for disadvantaged students into specific 
performance objectives. Assistance from a nonprofit firm was sought to oversee 
the competitive bidding process and to provide other technical services associated 
w*th preparing vnd obtaining a performance contract with a qualified commercial 
firm. 



The contractor will be paid according to the number of students who 
successfully complete the training program. He will be penalized for the number 
of students who do not achieve specified levels of performance. The school, not 
the contractor, selected the students. 

Performance contracts are not new to education. Elements of the notion 
were present in a variety of experiments during the 1920's and even before. But 
the concept of holding an educational agency accountable for results has not, to 
my knowledge, been put into practice before. 

This approach, if applied to the allocation of Federal resources and 
indeed all educational resources, would surely prove more economical than our 
present practices. Educational objectives translated into measurable student 
learning would offer a means for assessing program cost based on program effective- 
ness. When educational resources are directed toward achieving specific, measur- 
able outcomes, we may then be able to determine what resources are required to 
bring a given student to a given level of performance. 



The "independent educational audit" is another device being tested. 

Such an audit is directed simultaneously to determining if specified objectives 
have been met and to suggesting instructional procedures which might be more 
effective in meeting the objective. 

Independent fiscal reports have been applied to the fiscal side of edu- 
cation for many years. The "independent educational audit" is a process similar 
to that used in a fiscal audit. The emphasis, however, is on learning, on student 
performance as a result of financial outlays. One of its distinctive character- 
istics is a third-party review to assess the "truth" as seen by the independent 
reviewer without defensive, protectionist, or financial influence. The auditor 
attempts to make an objective accounting of what has happened to children's 
attitudes, skills, or knowledge in relation to locally established objectives and 
goals. Independent review is of fundamental importance in both business and 
government; its inclusion in the area of instruction is clearly in the best in- 
terest of education. The audit report would not only determine accomplishments 
but would also recommend procedures for getting results not attaint. " , Auditors 
would come from the ranks of professional educators, from among laymen, from 
universities, and private enterprise. Training is essential; in fact, we have 
already conducted one training institute to develop the concept. 

As you will recall, the Title III program was originally conceived as 
a way to provide risk money or developmental capital for implementing educational 
change in American education, particularly at the local level. I believe it is 
essential for this opportunity to be continued; educational accountability depends 
upon the availability of money for responsible investment in practices likely to 
produce improved results. 

Business typically budgets from three to fifteen percent of its funds 
for research and development, designed to produce better products, better ser- 
vices, more sales, or greater capability to produce these items. Until the 
passage of the Vocational Education Act of 1963 and the Elementary and Secondary 
Education Act of 1965, there was virtually no money earmarked for this general 
purpose in education. With the passage of these acts, approximately one- third 
of one percent of our educational resources became available for this type of in- 
vestment. 



With developmental capital set-asides, renewal can be directed through 
Federal, State, and local channels, and activity cai; be aimed at improving 
management leadership capabilities. The three sectors of government can work in 
conjunction with each other and with the private sector. 

At the Federal level, funds can be applied to "risk" investments. 

This is the onl^ governmental level which can commit the required dollars and 
manpower to accomplish research and development. Another major capability of 
the Federal level of government is identification of successful practices around 
the nation. Renewal capital can be used to determine the most pressing management 
and operational needs of school administrators and to identify successful school 
management and classroom practices that may meet these needs. A nationwide 
search should be organized to identify educational approaches which are effective 
and schools which have resolved major administrative and instructional problems. 

At the State level, developmental capital can be used to furnish in- 
centives toward achieving State priorities. Such an incentive payment program 
can help States meet their predetermined needs by concentrating funds on 



specifically identified and selected activities. 

At the local level, developmental capital might be used to secure a 
producer-consumer contract to share good educational practice. Pieces and parts 
of exemplary models exist throughout the nation. But we still lack an effective 
way on any major scale to insure diffusion, dissemination, adaption, and adoption. 
The producer-consumer contract development program would provide money for local 
districts to study, plan, train personnel, and install good practice which local 
communities want to obtain. 

Here's how it might work: 

After a local school district had completed a successful self-analysis, 
it wou]' 1 study model progr; *ns or practices in operation in other districts and 
select one which would be jst likely to meet its priorities. The "producer 11 
tahool district would be t ne district with the successful model program; the 
"consumer" district, the one seeking the solution. 

The Federal government would provide each State department of education 
with a complete, cataloged file of successful programs across the country. The 
State departments, having been involved in both the analysis and identification 
procedures at the local level, would identify producer school districts which had 
the potential of satisfying the needs of the consumer district and would help the 
consumer district decide which producer districts it wished to inspect. 

During the renewal process, the consumer district might be responsible 
for fulfilling two primary obligations: 

1. paying the producer school district a sum for each day of on-sitc 
examination. These funds might come from the consumer school districts renewal 
capital account and serve as incentive rewards for producer schools to continue 
good educational practices. 

2. conducting a simple evaulation for the other levels of government 
dur?.ng each on-site examination. This procedure can serve as a check-and-balance 
system to keep the "good practice" inventory up-to-date and valid. 

The culmination of the renewal process comes with the development of a 
performance contract between the consumer district and the selected producer 
district to reproduce the desired practice. 

Since seventy-five to ninety percent of local school system budgets are 
tied to salary, some mechanism for stimulating flexibility, creativity, and 
accountability is needed. Developmental capital has that potential. 

Finally, accountability may be enhanced in public education by the in- 
troduction of competition from either inside or outside a school system. This 
competition might be the product of fiscal incentives and a utilization of market 
mechanisms. For example, one way to obtain competition in compensatory training 
for disadvantaged students would be the creation of an escrow account. From this 
account, designated parents in the poverty category could obtain a voucher for 
use in accredited schools, according to performance stipulations. Let me describe 
a possible plan: 

A school system which receives Title I dollars might set aside an amount 



in an escrow fund. Vouchers totaling the amount of the escrow fund might be dis- 
tributed to parents of a selected number of children. The value of each voucher 
would be determined by a specific child’s need for compensatory education. 

Parents of the selected children might choose the educational agency 
to which they wanted to send their child for a portion of the day for compensa- 
tory training on a guaranteed performance basis. 

School systems or other suppliers of education services might negotiate 
individual contracts with parents, guaranteeing a specific level of achievement 
as requirement for payment. 

Through this negotiation, the parents and the schools would obtain in- 
centive leverage to give the children the truest and surest form of compensatory 
education. 



These are among the ideas that we have been working on at the U. S. 
Office of Education as ways of encouraging educational accountability. I look 
forward to discussing them with you in greater detail in the workshop periods 
which follow. If we are to fulfill the promise of ESEA Title III and make high 
quality education a reality for all American children, a move toward accounta- 
bility may be our next most important step. 
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by Thomas D. Clemens 



Perhaps it goes without saying that our concern today is with bringing 
about change which is relevant to valid educational and social goals and not 
with change just for the sake of doing things differently. It may also go with- 
out saying that there is a considerable amount of activity in educational set- 
tings today which does not contribute to better learning for children or to more 
effective or efficient operations of schools. This presentation, however, is 
based on the assumption that State education agencies are primarily concerned 
with quality education and, accordingly, any changes in schools will evolve from 
that primary concern. Having said that, let me now turn to some comments on how 
such changes can be facilitated. 

It is quite popular to assert that we know too little about how to 
bring about educational change and, no doubt, there is the same kind of continu- 
ing need for new knowledge about change processes as there is in any other area 
of human inquiry and endeavor. Still, it is important to point out that there 
is a substantial and rapidly growing body of knowledge about dissemination and 
utilization in all fields--some 4,000 separate reportsI/--and that a good part 
of this literature deals directly with educational change. Really lacking are 
concerted efforts to 'engineer* this knowledge into techniques, procedures, and 
guidelines for action. In a very real sense, the work of State agency personnel 
in general and dissemination personnel in particular is to engage in such engi- 
neering. 



In this presentation I will deal with four main points: 

I. The underlying assumptions of this discussion . (If you are to 
weigh my comments and determine their relevance to you, it seems important that 
you know what underlies these remarks.) 

II. What we know about change processes . (Although this portion of 
the presentation will focus upon change in educational settings, relevant infor- 
mation about knowledge utilization in other fields will be drawn upon, as appro- 
priate.) 



III. The resources of State education agencies as facilitators of edu- 
cational change (and you are better qualified to discuss this than I am). 

IV. And, lastly, possible directions of action by State education agen- 
cies in fostering goal-related changes in schools . 

Let us now deal with these four points. 
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I. Underlying Assumptions 



The first assumption upon which I am operating is that we see a good 
deal more educational change these days than we do education improvement. If we 
compare how our schools operate today with the way they did just a few years 
ago, it is evident that some kind of educational change is occurring; but if we 
are asked if children are learning more, or if our schools are responding ade- 
quately to the demands society places upon the schools, we are somewhat harder 
pressed. 



Secondly, it is assumed that our goal--yours in the State education 
agency and ours in the Federal government--is increased rationality in the de- 
cisions of local education agencies in adopting innovations which will improve 
attainment of educational goals. This may suggest the need for us to behave 
differently than simply to attempt to convince local schools that they should 
buy a product line which we endorse. 

Third, it is assumed that the State education agency is an essential 
agent for educational change in American education and is potentially the most 
powerful single change agency. 

The fourth assumption is that in education, as in all other areas, con- 
tinued effective utilization of knowledge for improvement requires deliberate 
efforts at linking practice or client groups to distant, more expert resources. 
This assumption dictates improved cooperative efforts by State and Federal agen- 
cies to diffuse and install appropriate innovations among and within local edu- 
cational agencies — efforts based cn clearly differentiated State and Federal 
roles, to reduce duplication and minimize possible gaps in needed functions and 
services. 



Fifth, it is assumed that the ultimate criterion of effective change 
is the maintenance and continued support of educational innovations adopted by a 
local school district. Put more crassly, the test is whether or not the adopter 
"puts his money where his mouth is." 

Finally, it is assumed that a continuing program for innovation and 
improvement in education requires something other than just more money. 

II ♦ What We Know About Change Processes 

One recent study of change in public school districts^/ indicates that 
there are three essential conditions for the adoption and continuation of what 
had been an. innovation. In the words of the investigators, these essential con- 
ditions are: 



• initiating mechanisms , that is the continuing flow of new knowledge 
into the school district from outside sources. This new knowledge may flow 
through print or other mass media, through training programs, through professional 
meetings, or through interpersonal communication with persons outside the school 
district. 



• sustaining mechanisms , or the capability of the school district to 
respond to and act upon such new knowledge. These mechanisms are within the 
school district, itself, and are relatively impervious to manipulation by out- 
siders. One of the most powerful internal conditions influencing the ability of 
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the school district to change is the degree of commitment and support for high 
quality educational programs within the community. Another very powerful factor 
is the degree to which the school district demonstrates 'openness/ or willing- 
ness to perceive problems and to make use of information from outside sources. 

A third powerful influence is the degree to which there is open communication by 
administrators and teachers about task-related matters. Yet another powerful 
predictor of innovativeness within the school district is the degree of flexi- 
bility there is in the school budget. Granted, a well-financed school is more 
likely to be innovative than one which is poorly financed, but a well-f inanccd 
school with a "locked-up" budget is usually less innovative than a poorer school 
district capable of re-allocating resources as needed during the trial of new 

practices .27 



a performance feedback transmissions , the clarity and measurability of 
objectives of the school district and the amount and clarity of communication 
about the degree of goal-attainment within the schools. Clearly related to this, 
of course, are similar communications about the effect of any trial innovation 
on goal attainment. 

From this conceptualization, it is evident that an 'outside agency,' 
such as a State department of education or the U. S. Office of Education, can 
play a highly significant role in communicating new information to school dis- 
tricts, regardless of any other conditions for change. For that reason, let us 
explore what is known about the adoption process and the function of information 
in that process. 

The best information on the subject has been collected by Everett 
Rogers-*/ and other persons studying the diffusion of innovations through social 
systems. What this extensive body of literature (largely empirical) tells us is 
that rational adopters of innovations tend to go through a series of five stages 
culminating in adoption and that it is rare for any one of the stages to be 
skipped. Rejection of an innovation may occur at any point in the process. 

These stages are: 

Awareness , in which the potential adopter learns of an innovation, or 
of some alternative to his current practice; 

Interest, a stage in which the more cosmopolite and innovative of the 
potential adopters seek out additional, more technical information about the 
innovation; 

Evaluation , during which the potential adopter makes an f in-the-head 1 
assessment of the relevance of the innovation to needs, to apparent advantages, 
and to potential problems for him; 

Trial , ir which the potential adopter (again, a smaller number of the 
more innovative members of the social system) makes an actual field test of the 
innovation on a limited basis. The least innovative members of the social system, 
the 'laggards, 1 as Rogers calls them, are likely to skip this stage, since they 
can see the benefits that adopting the innovation brings to the majority of their 
peers who have already adopted the innovation. 

Adoption , the final stage, when the adopter decides to make full-scale, 
continuing use of the innovation until it is replaced by something even better. 
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Of particular importance in fostering the spread of an innovation is 
the fact that potential adopters tend to use different information sources at 
different stages of the adoption process. Thus , using the Ross and Halbower 
terminology, it is necessary to use a variety of initiating mechanisms in order 
to stimulate and spread the adoption of a new practice or product. The types of 
information sources used in the various adoption stages are summarized in Figure 
1, following. 



Sources of Information Used in the Adoption Process 



Adoption Stage Information Source 

AWARENESS MASS MEDIA 



INTEREST 



INTERPRETED, TECHNICAL INFORMATION 
INTERPERSONAL COMMUNICATION 



EVALUATION 



TRIAL 



INTERPERSONAL COMMUNICATION 



DIRECT EXPERIENCE 
INTERPERSONAL COMMUNICATION 



ADOPTION DIRECT EXPERIENCE 

INTERPERSONAL COMMUNICATION 
LIMITED TECHNICAL INFORMATION 



Figure 1. 

As you see, awareness of new practices and materials usually comes from 
exposure to the mass media and, as might be expected, the more innovative and 
cosmopolite members of a group are the first to become aware, since they tend to 
read more, to reach out toward outside information sources, and generally to 
look outside their own social system. 

Those who are sufficiently aware and interested to seek more informa- 
tion about the innovation then turn toward more technical information sources 
which provide more detailed information, such as government bulletins, integra- 
tive reports, or expert advice. The more innovative the adopter, the more likely 
he is to use technical information sources; the less innovative, the more heavily 
he will rely on interpersonal communication, particularly with individuals he 
knows and believes to have more expert knowledge. 

During the pre-trial evaluation stage, all potential adopters rely most 
heavily on interpersonal communication with earlier adopters^ The very early 
adopter may use expert consultants, or change agents such a3 the county agricul- 
tural agent, but the important point is that during the evaluation stage, adopters 



prefer to learn from people rather than from documents. 

In both trial and adoption stages, the direct, first-hand experience 
of an adopter and the advice, assistance, and encouragement of expert consultants 
or even earlier adopters are the most important information sources. Clearly 
then, our attractive, four-color brochures and individual reports of projects, 
so popular among most educational dissemination personnel, are useful for little 
more than making others aware of the existence of an innovation. Such communi- 
cations are unconvincing for those same potential adopters as they move closer 
to evaluation, trial, and adoption of the new practice, and may even mislead 
naive persons. 

There are a number of other generalizations we can draw from the liter- 
ature on dissemination and utilization of knowledge in fields other than educa- 
tion, but, in the interest of saving time, let us turn our attention to some of 
the distinctive characteristics of educational change. Ronald Lippitt^/ has 
pointed out that there are two clear differences in programs of social change, 
such as education, when compared with change in the physical or biological sci- 
ences . 



The first of these is that educational changes are rarely limited to 
’thing changes , 1 but are usually 'people changes.' Put in less cryptic form, 
this means that most innovations in medicine and industry simply displace one 
product or practice with a newer, better one; but in education the central 
changes occurring in most innovations — team teaching, non-graded organization, 
even new curricula — are changes in role perceptions, attitudes, and behaviors of 
the adopters or practitioners. Quite clearly, a teacher who sees no change in 
her role, or who does not change her behavior when she leaves the self-contained 
classroom to engage in team teaching, is going to be very frustrated, very in- 
effective, or both. Some of the major problems in the instructional use of 
television have resulted from the fallacious assumption that television can do 
all sorts of wonderful things without any change in the operation of the school 
or the behavicr of the teacher. 

The second difference to which Lippitt pointed is that most educational 
changes are not simple adoptions of a solution developed elsewhere, but adapta- 
tions of the school to the innovation and of the innovation to the school. This 
means that it is vital to know both the essential conditions for the innovation 
to succeed and the unchangeable constraints within the school, so that any a- 
daptation made in the innovation may actually result in educational improvement, 
rather than in maladaptation. A fascinating description of the maladoption of 
an innovation can be found in the last chapter of Carlson's "Adoption of Educa- 
tional Innovations."^ 

If a State education agency is to facilitate spread of improved prac- 
tice, it is essential that it tailor its strategy and activities to the actual 
conditions and constraints in the local school districts. You people know these 
constraints better than I, but allow toe to suggest a few which are particularly 
vital in programs for change. 

Perhaps the first immutable constraint that influences educational 
change is the pluralistic nature of American public education. It is a truism 
in organization theory that complexity and diversity in any operation impedes 
rapid movement of the component parts of the organization toward a common goal.Z^ 
Conversely, of course, pluralization permits differential response by parts of 




40 



the organization to diverse conditions in various parts of the client population. 

It is obvious that American education is (and ought to be) pluralistic, since 
ours is a large, heterogeneous society which is unlikely to be served adequately 
by a monolithic educational system. Hie tradition of local control of education 
does guarantee some responsiveness of the educational system to local require- 
ments and the primacy of State responsibility docs assure a concern for social 
requirements above those of any single local community. Further, legislation at 
the national level in recent years suggests some national educational goals 
accompanied by ■'afeguards for the protection of State and local requirements. 

Put briefly, the pluralistic system of education we have *n this coun- 
try appears to conform to American values, but it does make the problem of system- 
wide change slower and more difficult* This dictates closer working relationships 
and more explicit differentiation of functions at the State and national levels. 

A second major constraint to educational change in the United States is 
the diffuse, inexplicit nature of educational goals.—/ One of the most powerful 
predictors of adoption of an innovation is the relative advantage of the innova- 
tion over current practice. When the goals of an organization are not stated in 
clear, measurable terms and when there are conflicts among goals, i- becomes 
difficult to identify any clear advantage in any single practice over any other. 
This suggests the need for more precise, operational, and attainable educational 
goals if the pace of educational improvement is to be accelerated. The State 
education agency is the most appropriate and likely source of leadership for 
overcoming this constraint; but let us recognize that explicit goal definition 
is, in itself, an educational innovation. As a result, we may anticipate a 
somewhat slower spread of educational improvement than improvement in other 
areas of human endeavor. 

Still another constraint upon educational change is our limited scientif- 
ic tradition in education. It has been said, by a person whose na^e escapes me*., 
that human endeavors may be plotted between two poles: experience-based practices 

and science-based practices. It seems rather clear that most educational prac- 
tices fall far toward the experience-based pole. Until there is increased use 
of the expanding knowledge base in education to carry on educational programs, 
it seems doubtful whether our schools can expect to work continuously toward 
ever- improving educational practices. There is need, then, for more effective 
communication of verifiable knowledge to educational practitioners in forms that 
are both understandable and relevant to their work. Likewise, there is need for 
increased efforts in both pre-service and in-service education to increase the 
competence of educational practitioners in using and assessing scientific know- 
ledge. 



Still another coasts aint in educational innovation lies in the fact 
that there is no single locus for making and implementing educational decisions. 
Remember that an innovation is likely to be adopted if the adopting unit--be it 
a person or an organization — perceives greater relative advantage through adoption 
of the innovation than in continuing current practice. In most areas of endeavor, 
it is possible to identify a single decision-making locus: in agriculture and 

medicine, this locus is a point, the individual farmer or physician; in industry 
and in military organizations, the decision locus is a line, the command or mana- 
gerial line. Compare this with education, where most changes of any significance 
involve a number of congruent decisions in different loci: policy decisions by 

the s chool board, management decisions by superintendents and principals, sub- 
ject matter decisions by supervisors, and implementing decisions by each classroom 



teacher involved. However, one fact stands out: unless the superintendent, who 

can allocate resources and provide rewards for innovative behavior, is supportive 
of an Innovation, it will not be installed. Thus, it is essential that su^erin- 
tei;Hr»r.cs of schools be provided information and assistance which will help them 
make rational choices among alternatives. Efforts must also be made, of course, 
to communicate with and assist the decision-makers and implementors at other 
levels, who are necessary participants in the adoption process. If such assf st- 
ance is not provided, the superintendent may believe he is leading a parade down 
the street to improved educational practice, only to find that even the band is 
not following him. 

III. Resources of State Education Agencies for Change 

Let us turn briefly to a consideration of specific characteristics and 
capabilities of the State education agency as a facilitator of change. Here, 
again, there is no doubt that you who are State department officials are more 
knowledgeable than I about this question, but it may be useful to Bee yourselves 
through the eyes of an outsider. 

Underlying all other considerations, the statutory position of the 
State as the responsible agent for education in the United States dictates a 
leadership role in educational improvement by the State education agency. Two 
underlying implications of this central role of the State agency in determining 
and implementing educational policy deserve explicit discussion. First is the 
obvious fact that quality education requires a certain critical mass of resources 
if there is to be real equality of educational opportunity for all our children. 
With the wide divergence in tax base among school districts, the State is the 
most natural governmental unit to serve as the planning base for education, even 
when individual needs and characteristics of local schools are honored. Secondly, 
it has been arguedi^ that local school programs must necessarily reflect the pre- 
vailing norms of the local school district, with the possibility that the needs 
of all members of the community will not be recognized and dealt with. For example, 
we ci,n identify communities where eighty-five percent or more of the public school 
graduates go on to higher education. The objectives and curriculum of the schools 
in this community are likely to stress academic excellence over all other possible 
educational values. Yet the fifteen percent minority in that community may well 
be part of the majority of public school students of the entire State. The only 
assurance that their needs will be served equitably is the mediating influence 
that State educational policy may have on local norms. 

A second major resource of the State agency is its unique position to 
provide educational leadership to all public schools. Perhaps one of the most 
salutary changes in American education in the past few years has been the degree 
to which State agencies have recognized and assumed their responsibility for a 
leadership role. This role is a natural one, both because of the State's role 
as the responsible governmental agent for educational policy and because it is 
in a more favorable position than individual school districts to draw upon external 
resources for educational improvement. Let me also suggest that the degree to 
which the State agency is assuming its leadership role is reflected primarily in 
the number, quality, and character of its coimrunications on professional matters 
to educators, the public, and the legislature which it serves. 

This brings us, of course, to a third major resource of the State 
agency as a facilitator of change: its unique position as a source of expert 

advice and staff service to the legislative and executive branches of the State 
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government. There is evidence that State legislatures view the State agency as 
a major, in many cases, the major, source of assistance in formulating educa- 
tional policy.—' Through the types of communications it provides to the Gover- 
nor and the State Legislature, the State agency can assure the development of 
sound, rational State policy on education and provide the intellectual basis for 
the policy makers to make fiscal and other decisions which will permit implemen- 
tation of that policy. 

A fourth unique resource of the State agency in facilitating change is 
one which is usually perceived as an inhibitor of educational change--its regu- 
latory function. Let me suggest that although regulation can be performed in a 
manner which is deadening and inhibiting, it can also be used as a means of en- 
couraging educational improvement. Assuring the presence of qualified teachers 
is a regulatory function but one that should assure better, not worse, education. 
If certification requirements are used as a means of preventing trial and assess- 
ment of staff differentiation in the schools, this can inhibit educational change; 
however, such inhibition results more from the way the regulation is implemented 
than from the standard, itself. The regulatory function can also be used to 
encourage more explicit statements of educational goals at the local level and 
to assure justification of proposed new courses of action. If exercised in this 
way, it is possible to develop procedures for regulating educational improvement 
rather than for blocking it, and to have guidelines for further waiving of regu- 
lations under appropriate conditions, or even for amendment of regulations in a 
manner consonant with changing educational needs. 

Finally, of course, the role of the State agency in educational finance 
should be a powerful instrument in fostering educational change. The role will 
not be used in this way, however, if all financial transactions between State 
and local educational agencies are based on formula grants. Most innovation in- 
volves some degree of risk-taking. It requires the potential for acquiring or 
re-allocating money for trial and installation of changes. The State can facili- 
tate change by providing risk capital, fiscal flexibility, and reward for efforts 
to improve educational programs. 

IV. Implications for Action 

In the final analysis, the implications of the issues we have discussed 
relating to actions you might take are limited by the resources, constraints, and 
values of your own State. I suggest, however, that you may be able to define 
your unique role as a facilitator of change if you consider how the resources 
available to you can be applied in assuring the presence of the three essential 
conditions for installation of educational innovations. This may be viewed as 
the matrix shown in Figure 2, next page. 
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State Education Agency Resources and Essential Conditions for Change 



Conditions for Change in the Schools 

Performance 



New Knowledge Capabilities Feedback in 
to Schools of Schools Schools 



SEA Resources 
1. Statutory Role 


i 






2. Communication - Leadership 








3. Staff Service to Legislature and 
Executive 








4. Regulatory 








5 • Finance 


! 







Figure 2. 



In the function of transmitting new knowledge to schools, the State 
agency can have the greatest and most immediate impact on local change processes, 
first, because of the State agency's intermediate position between local and na- 
tional resources and, second, because such a function can be performed independ- 
ently of the local district. 

To perform adequately in transmitting new knowledge to the schools at 
times and in forms which are understandable and usable, the State agency must 
first have the capability of taking in, processing, and then communicating a con- 
tinuously growing body of research, theory, and reports of good practice in 
education. This can be done best by establishment of a State agency information 
center which links distant resources to local needs. Without such a center. 

State dissemination efforts can only be intermittent, unfocused, and partly 
effective. 



Unfortunately, we are not doing very well in systematically receiving, 
processing and communicating valid, needed information. In a recent study of 
educational information services in education, it was found that only thirteen 
State departments of education operate any kind of systematic information ser- 
vice.—' Even though, through the Educational Resources Information Center 
(ERIC), educators can acquire as much as seventy percent of the current body 
of educational research and related information, few school district personnel 
outside urban centers or university towns have direct access to ERIC materials. 
Further, there are few State education agencies or local school districts which 
systematically feed into the ERIC system reports of their own experience which 
have utility for other educators. 
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Let us consider what the characteristics of an effective State educa- 
tional information system might be. In order to do so, it is essential to con- 
sider first of all how the educational community behaves as information users. 

To begin with, educators, like practitioners in other fields, tend to be apathetic 
about seeking out new knowledge. There is evidence that practitioners make 
first and most frequent use of the most accessible information source, even when 
they believe that the source does not provide very good information .—' It fol- 
lows then, that unless accurate information is at least as tccessible as in- 
accurate information, it will be inaccurate information which will reach educa- 
tors. Thus, information services must be as close as a local telephone call. 



We also know that educators, like other practice groups, only more so, 
prefer interpersonal communication to print or other mass media.l-t/ > ± 1 ' Indeed, 
the most likely effect of providing teachers or administrators vast quantities of 
materials to read is that they will ignore what has been provided them. In a 
recent study of the characteristics teachers and principals want of their infor- 
mation systems ,1^/ they ranked the desired characteristics of an information sys- 
tem as shown in Figure 3. 



Information Service Characteristics Preferred by Educators 



Rank Characteristic 

1 Easy Access to Information 

2 Currency of Information Provided 

3 Evaluation of Material Provided 



4 Comprehensive Coverage 

5 Rapid Response to the Query 

6 Thorough Documentation 

7 Flexibility in Amount of Detail 



Figure 3. 

One might say, on the basis of these data, that unless we provide comprehensive, 
up-to-date, and screened information to educators, rapidly and with minimal ef- 
fort on their part, we cannot expect to have them use any information service 0 



We have already discussed the information sources used by potential 
adopters of innovations as they move through the adoption process. Recently, 
Havelockit^ has stated that not only different formats but different types of 
information are required in the process of adopting and installing an innovation. 
He summarizes this under the acronym, DAETETM. 
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Information Requirements for Rational Adoption of Innovations 



Stage 

Diagnosis 

Awareness 

Evaluation (Pretrial) 
Trial 

Evaluation (Post trial) 
Installation 

Maintenance 



Questions 
How am I doing? 

How can I do better? 

Will it work for me? 

How can I do it? 

How did I do? 

How must I change? 

How do I continue? 



Needed Information 



Current performance data 

Alternative solutions to my 
problem 

Settings and conditions where 
the innovation succeeded 
(failed) 

Sources of expert assistance 
How to adapt to local conditions 

Comparative effectiveness of 
old and new practices 

Needed resources and actions 
to make the innovation work 
operationally 

Resources and actions needed 
to continue and modify the 
innovation 



Figure 4. 



So, if the State agency is to facilitate effective utilization of 
knowledge in improving education, it must be prepared to tailor information both 
in format and content to the information requirements of persons having different 
educational roles ana it different stages of the adoption process. Unless its 
program is based upon continuing assessment of the characteristics and needs of 
its users, it will be a waste of money and effort,. 

An effective and responsive information system will have to engage in 
a number of different functions if it its to provide the products and services 
needed by the system users. Let us consider each of these elements briefly. 

The function s of the information system will include: 



Acquisition : The process of collecting and screening information. 

Processing : Indexing, abstracting, and other treatment of the in- 

formation, so that it may be stored and retrieved 
effectively and efficiently. 



Storage : Maintenance of the file of information, so that it is 

readily accessible. 



Search and 

Retrieval: Methods of locating in the file the most highly relevant 



46 



Information 
Analysis : 



information required in a rapid, efficient manner 



The tailoring of the raw information retrieved to maximize 
its relevance and utility for the user* 

Dissemination : Providing the information to the user in the format 

':ha'„ best meets his requirements. 

The products of such an information system may either be basic products, 
or derived products. Basic information products are those which are provided to 
the user more or less in the form in whi.h they were acquired. Examples include 
data, research reports, and reports of current practice* Derived products are 
those resulting from treatment of the basic information in forms to make them 
most useful for the user. Whereas graduate students and researchers may be 
satisfied with basic information products, most practitioners are more likely to 
need derived information products. Examples of these include abstracts, for 
brief descriptions of individual documents; information analyses which integrate 
the content of many pieces of information to indicate broad findings, generaliza- 
tions, gaps in current knowledge, and so forth; and referral lists, which indicate 
to the user where he can find more information, sources of expert advice, and 
sites where he can observe a new practice in action. 

The services required by most users include: query negotiation , in 

which the agent of the information system attempts to specify the information 
need more precisely, thereby spreading the search and the increasing relevance 
of the information provided; outpu t screening , in which the retrieved information 
is examined to assure its relevance to user needs; and client briefing , in which 
the agent of the information system consults with the user to determine whether 
his needs have been met and if additional efforts are required. 

Since effective use of a modern information system usually requires 
qualifievi information specialists assisting the user, and since educators rely 
so heavily on interpersonal communication, it is doubtful whether any but the 
smallest States can expect to have a single information center serving the needs 
of all educators. As a result, it is quite likely that a fully matured educa- 
tional information system for a State will have a State information center co- 
ordinating and serving a series of local information centers located to serve a 
number of contiguous small districts or one large urban school district. The 
model of regional service centers or boards of cooperative educational services 
now exists and only waits to be adapted to provide information, services, as well 
as the other services they now provide. 

No doubt, this all sounds terribly expensive and complex. It is, but 
is such a system to assure that educational decisions are based on the best 
current information any more expensive than the human and financial costs of 
educational changes which fail to improve our schools? Agreed, the costs of one 
such system in each of the fifty States boggles the mind, but through inter-State 
coordination and differentiation of functions at State, local, and national 
levels, it is possible to provide highly sophisticated information services in 
both an effective and cost-beneficial manner. 

Let us explore how this might occur. First, through use of the ERIC 
system, the State agency can provide access to a current bank of over 30,000 
separate documents which is growing at the rate of 1,000 per month. By using 



an ERIC-generated product, the Current Index to Journals in Education , published 
by Crowell, Collier, and MacMillan, it is also possible to search an index to 
articles in some 200 journals, updated monthly. In addition, the School Research 
Information Service of Phi Delta Kappa provides access to reports of current 
practices in schools. 

What is missing for a complete national file is ready access to docu- 
ments generated in all school districts and all State agencies. If each State 
agency were to set up a campaign for continuous collection of documents generated 
within the State, screen out those of purely local or State interest, and feed 
those of more general utility into ERIC or SRIS for processing and announcement, 
it would contribute to the national hank of educational information without hav- 
ing an overwhelming information processing burden of its own. It could also 
benefit from having access to all the information generated in other States. 

In like manner, the Office of Education, through the ERIC clearing- 
houses and a special research interpretation program called the Targeted Com- 
munications Program, is generating information analyses on problems of signifi- 
cance to educational practitioners. As you may know, the reports generated in 
the Targeted Communications program are repackaged into PREP (Putting Research 
Into Educational Practice) Kits, which are provided to each State agency for its 
use. The State agency may, if it wishes, simply announce the availability of 
PREP Kits and make copies available to interested educators; but it may also 
adapt these materials to fit more closely to State and local needs and make them 
part of their own dissemination program. 

Through such efforts as these, a State educational information system 
can build upon national resources and focus most of its efforts on information 
search, retrieval, and user services for its own clientele. A central State 
agency information center can provide direct services to its own personnel and 
to elected State officials, and serve as the locus for planning and pilot opera- 
tion of a service prior to establishment of local information centers* When 
such centers emerge, funded, perhaps with Title III or other Federal monies, the 
State agency role moves from exclusive emphasis on operations to one of coordina- 
tion and leadership* 

Since this presentation has been going on for an inordinate period of 
t/me, let me simply plot out some of the factors relating to how the State agency 
may increase the innovative capabilities of local schools and facilitate improved 
needs assessment and goal attainment, leaving it to you in your discussions to 
put flesh on these bare bones* 

Since we know that community concern for and participation in edu a- 
tional dialogue enhances the innovativeness and responsiveness of schoolsi§^ the 
State agency, in its leadership role and in providing staff services to elected 
State officials, might well place emphasis on the need for effective two-way 
communication between the school and the community and may in time look toward 
regulations which mandate increased efforts toward such communication. 

Both in its administration of Federal funds and in State financial 
policy, it may be possible to provide 'risk capital* for innovative programs as 
well as incentives for local schools to seek changes which lead to educational 
improvement , In such programs, it is highly advisable to require commitment of 
local funds to the innovative effort from the very beginning, with increasing 
allocation of local resources as the project progresses. There are data both in 
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education—' and outside — ' that an innovative program which involves the use of 
local monies from the beginning is more likely to be installed than one which 
receives full funding from outside sources. 

Again, since openness to new knowledge and to statements of need are 
gcod predictors of innovativeness, encouragement and recognition of such openness 
by State agency personnel may do much to increase a school district's innovative- 
ness • 



In order that we do not get lost in specialized jargon, let me rai'erate 
that Rosb and Halbower use the term, fostering performance feedback, to mean the 
degree to which an organization specifies its goals in measurable terms raid com- 
municates to its members about the degree to which goals are being attained. 

Both as a leader and a regulatory, the State- education agency can en- 
hance improved performance feedback transmissions within and between school dis- 
tricts. As a leader, the State agency can do this by communicating to local 
districts about means of defining goals operationally and assessing the degree 
of their attainment. In so me cases, it may even be possible to stimulate State- 
wide participation in development of model educational objectives. As a regu- 
lator, the State agency may require operational specification of objectives in 
proposals for Federal or discretionary State funds (as I understand Utah is now 
doing), it may even require by regulation that participation in certain State 
programs be based upon operationally-stated objectives, or it may even offer in- 
centives for provision of undisputable evidence of satisfactory performance. 

Finally, since evidence of performance is quite understandable and 
attractive to elected officials, the State agency, itself, should shape its com- 
munications to the State legislature in performance-oriented terms whenever 
possible, and stand ready to assist in formulation of legislative proposals for 
improved goal definition and performance assessment in education. 

At last, to the relief of all of us, I shall close. This presentation 
has ranged widely and overlong. If I am to leave you with any coherent thought, 
it is this: 

The State department of education is potentially the most powerful 
agency for educational improvement in American education. The degree to which 
it reaches its potential is contingent upon the degree to which it exercises its 
full resources for facilitating change and the degree to which it bases its 
efforts on full, application of the extensive body of knowledge on means of 
spreading and installing educational change already availaole to us. One of its 
most viable means of having immediate impact is through improvement of communi- 
cation of new knowledge to local schools. In this endeavor the Office of Educa- 
tion stands ready to participate as a partner. 
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by V. James Popham 



In the shadow of dramatic advances in the field of instructional psy- 
chology, we educators have been quite properly reminded how imperative it is that 
this increased instructional sophistication be directed toward defens ible educa- 
tional outcomes. In the last several years, particularly as a consequence of 
ESEA Title III program requirements, the phrase needs as sessment has been employed 
to describe that operation which is designed to identify those areas of education- 
al deficiency most worthy of amelioration. 

Needs Assessment Defined 

Since this discussion will focus on specific problems and solutions 
associated with the conduct of educational needs assessments, a definition of 
such operations is warranted it the outset. Briefly, educational needs assess- 
ment is a technique for identifying those educational objectives which most need 
to be accomplished in a given instructional situation. The conception of an 
educational need in this context is a standard one. First, a desired learner out- 
come is identified. Second, the learners 1 current status with respect to that 
outcome is ascertained. The difference between the current status and the de- 
sired status is considered to be an educational need. This conception is repre- 
sented in Figure 1. 
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Figure 1. 



Once having identified a number of educational needs, the most diffi- 
cult task of the needs assessor is to rank these in some way so that the educa- 
tional system can be directed toward the satisfaction of the most important needs. 

There has been growing acceptance of the view that to adequately deter- 
mine the learners* educational needs we must be attentive to a wide variety of 
educational outcomes, rather than to only the customarily sought types of intel- 
lectual achievements. As a consequence, those working in the needs assessment 
arena are now urging educators to consider the identification of needs with 
respect to objectives in all three domains of learner behavior, that is, the 
affective, cognitive, and psychomotor. By cognitive, of course, we refer to 
intellectual types of learner outcomes. Affective needs pertain to attitudinal, 
valuing, or emotional types of learner outcomes. Fsychomotor needs are associa- 
ted with a learner’s physical and motor skills. 
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Yet, while this general approach to needs assessment has received sub- 
stantial support from those individuals actually engaged in such activities, 
anyone who has attempted to implement this general strategy has discovered that 
the job is not simple. There are several very thorny problems which must be re- 
solved, whether the needs assessment is conducted at a national, state, or local 
level. 

Proble m Number One : Identifying Educational Preferences 

In establishing what we wish our students to become, it must be recog- 
nized that this operation is exclusively one of valuing. There are no formulas 
which, if implemented, would obviate the necessity of someone reaching judgments 
regarding what educational goals ought to be established. But even recognizing 
that one's values are usually held with varying degrees of defens ibility, we 
still encounter a number of practical difficulties in determining value prefer- 
ences regarding educational goals. This is complicated, of course, because of 
the diversity of groups who might wish to influence the establishment of desired 
educational outcomes. For example, let us assume for the moment that those con- 
ducting an educational needs assessment identify parents as Individuals who 
should have a voice in the establishment of educational goals. How are such 
parental preferences identified? Does a staff member from a needs assessment 
project interview parents and ask them, in essence, "What do you want your chil- 
dren to be like at the end of their schooling?” One suspects that responses to 
this general question would be given at such different levels of generality that 
it might be impossible to categorize parental preferences in a meaningful manner. 
Similarly, if academicians (or any other group) were to be polled, what practical 
methods exist for getting definitive, manageable statements from those individuals. 

Although several needs assessment groups have tried to approach this 
preference identification operation faithfully, the procedural requirements of 
securing an adequate set of preference data from potential contributors have not 
been resolved. 

Problem Number Two : Identifying the Learners 1 Status 

A problem of equal difficulty involves determining the learners 1 current 
status with respect to a variety of educational outcomes. Through the years, 
educators have been inclined to use standardized achievement tescs for determin- 
ing the learners 1 current abilities. Recent advances in measurement circles, 
however, suggest the thorough inappr opr lateness of using typical standardized 
tests for such assessments. The unsuitability ol such tests rests upon a basic 
distinction between norm- referenced and criterion-referenced approaches to 
measurement.^ A standardized test is, generally, a norm-referenced test and is 
designed primarily to identify an individual^ status with respect to a norm 
group, that is, other individuals who have completed the same test. Because of 
tho necessity to produce variant scores, scores which permit comparisons among 
individuals, standardized tests are often unable to represent the complete range 
of learner behaviors which we need to know about. 

Criterion-referenced tests, on the other hand, are designed to measure 



^Pophain, W. J., and Husek, T* R. "Implications of Criterion-Referenced Measure- 
ment." Jo urnal of Educational Measurement , Volume 6, No. 1, Spring 1969. 
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a learner's status with respect to a specified performance standard and, as such, 
arc more suitable for purposes of needs assessment. Unfortunately, criterion- 
referenced tests do not exist in quantity. No established commercial test dis- 
tributors have developed a sufficient number of criterion-referenced tests to be 
of any real utility to a needs assessor, and without such tests one cannot ade- 
quately measure the learners 1 current status regarding the outcomes in which we 
might be interested. 

Probl em Number Three : Contrasting Preferences With Status 

Referring back to the general model for identifying educational needs 
(Figure 1), it is important to note that learners' current status must be con- 
trasted with (usually subtracted from) desired learner outcomes in order to de- 
termine an educational need. Particularly because of the unsatisfactory methods 
being used to establish educational preferences and learner status, these com- 
parisons are not easil> produced. For example, what happens if a group of busi- 
nessmen respond in rather general terms to a needs assessment interviewer that 
they are looking for young men and women who can perform different kinds of 
clerical tasks, while actual data regarding learner post-high school clerical 
competencies are ^ery specific . Comparisons are difficult, if not meaningless. 

The procedural problems of contrasting current learner status with preferred 
learner outcomes are very real and have not yet received sufficient attention 
from those involved in educational needs assessment. 

Collections of Ins tructional Ob jectives and Related Measures 

As can be seen from the three problems previously identified, the pri- 
mary difficulties of implementing the usual needs assessment model are procedural. 
While we may have the wisdom to devise a general strategy, the priory technical 
deficiency rests on inadequate vehicles for securing the right kind of data. 

One significant advance in recent months has been the establishment of 
an agency which is attempting to collect large numbers of instructional objectives, 
stated in measurable t^rms, plus sets of devices to measure each of these objec- 
tives. With the existence of such collections of objectives and items, some 
procedures for dealing with each of the three problems identified above can be 
devised. A systematic approach to needs assessment employing these objectives- 
items collections will be described in the remainder of this presentation. 

Before turning to those procedures, however, the principal agency where 
these objectives and items are being collected should be briefly described. The 
Instructional Objectives Exchange, established by the Center 2 for the Study of 
Evaluation at the University of California, Los Angeles, was created approximately 
eighteen months ago in an attempt to serve as a national depository and develop- 
mental agency for sets of operationally stated objectives and for measuring de- 
vices which could be used to assess the degree to which such objectives had been 
achieved. The Instructional Obje^civas Exchange (IOX) is currently collecting 
and, as its resources permit, making available to any educator collections of 



o 

The Instructional Objectives Exchange was established as one component of the 
Center's Project for Research on Objective Based Evaluation (PROBE). The activities 
of PROBE are guided by the following committee: Marvin C. Alkin, Eva L. Baker, 

Madeline C. Hunter, Ron G. Mclntire, W. James Popham, and Rodney W. Skager. 
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objectives and measuring devices in a number of fields. 

Wishing to see many teachers use its services, that is, wishing to have 
more teachers identify their objectives in measurable terms, IOX is currently 
distributing objectives according to commonly used subject matter and grade level 
categories. One might secure an IOX Collection of mathematics objectives in grades 
K through three or reading objectives grades four through six. Similarly, a 
collection of objectives might be prepared for a senior high school geography 
course or a junior high school woodworking course. At a later point, it is antic- 
ipated that IOX objectives will be reclassified according to categories which 
may be more functional than typical subject-grade boundaries *3 in every case, 
insofar as its resources permit, the Exchange is distributing with its objectives 
ser.s of test items or other procedures for measuring those objectives. The ex- 
istence of such collections and related measures makes possible a systematic 
approach to needs assessment which will solve the major problems identified above. 

Three Domains 



When one considers a three-domain attack upon educational needs assess- 
ment, this IOX service becomes even more critical. There are relatively few in- 
structional objectives available, even as models, which deal with affective 
learner behavior or with certain kinds of cognitive and psychomotor behaviors. 

The Exchange is attempting both to develop and to pull together tnose scattered 
sets of such goals so that exemplary collections of hard-to-devise objectives 
will be available- More importantly, perhaps, measuring devices for assessing 
attainment of such objectives will be produced * 

To illustrate, it is well known that simplistic measurement approaches 
cannot be used in gauging a learner 1 s attitudes. Sophisticated approaches must 
be employed which eliminate cues as to how the learner n should 11 respond. Simi- 
larly difficult is tapping high level cognitive abilities, such as analysis and 
synthesis. Equally taxing is the measurement of psychomotor behavior where sub- 
jective appraisal is required, such as in judging a student 1 s gymnastic perform- 
ance. We cannot expect each separate needs assessment project to have the re- 
sources or the expertise to develop such objectives and measures. The task is 
too immense# It is precisely for this reason that the Instructional Objectives 
Exchange was established. 

Problem Number One Solved 

By presenting sets of measurably stated objectives to the potential 
reference groups, that is, the groups of individuals whom the needs assessment 
staff wishes to involve in its survey, the needs assessor can derive a systematic 
set of preferences because the sets of objectives can be presented in a relatively 
constant form to the various groups. For instance, let us suppose that we wish 
to involve representatives of (1) the community » (2) the learners , themselves, 
and (3) educators in establishing the desired goals of an educational system. A 
set of objectives from the IOX collection, each perhaps accompanied by a sample 



^The development of alternative classification schemes is being supported by 
a contract with the U. S. Office of Education, Division of Comprehensive and 
Vocational Research. This contract is also supplying considerable support for 
the collection and development of IOX objectives. 
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measurement item to more clearly communicate the nature of the objective, would 
be presented to representatives of each of these three groups. It might be 
necessary to modify the language of the objectives somewhat for the different 
groups, for it may not be realistic to expect young learners and certain com- 
munity representatives to understand the technical language which would be com- 
prehensible to educators. Even with such modifications, however, the basic set 
of objectives would remain the same. 

We could ask these groups to appraise the objectives in a variety of 
ways. A very simple approach, for example, might ask the representative indi- 
viduals to rate or rank each of the objectives for purposes of its possible in- 
clusion in the school curriculum. Rating? might be supplied according to the 
degree of importance the individual attached to the objective,. These ratings, 
for example, supplied on a five-point scale, could be used in developing average 
estimates of the importance attached to each objective by the particular group 
involved. The preferences of the several groups could be arrayed in a relatively 
simple manner which would permit comparisons among the average ratings for each 
objective and, of course, the identification of those objectives considered most 
important by all groups. Similarly, objectives considered unimportant by all 
three groups would, once identified, undoubtedly be eliminated from further con- 
sideration. In instances where there was significant disagreement among the 
participant groups, further exploration among group representatives might reveal 
the reasons for such disagreement and, possibly, a way to reconcile the disparate 
ratings. On the other hand, in such instances of disagreement it is perfectly 
reasonable to attach more weight to the preferences of one group than to another. 
This is a philosophic issue which would clearly have to be faced by those con- 
ducting the needs assessment. 

Problem Number Two So lved 

As indicated before, the use of norm-referenced tests for the assess- 
ment of current learner competencies is impermis sable. However, the existence 
of pools of test measures foi the objectives which are being used makes possible 
the ready preparation of criterion-referenced tests. A set of such items, ran- 
domly drawn from the available pools, could be administered to the learners and 
data could be secured regarding the degree to which the learners were able to 
master the objectives. These measuring items are not necessarily designed to 
produce variance among learners. They are simply designed to be congruent with 
the objective and, as such, represent the most adequate reflection of the 
objective's attainment. The avoidance of standardized tests and the use of such 
criterion-referenced test item3 (for each objective indentiiied as desirable, on 
the basis of preference data) will yield a far more sensitive reflection of 
current learner status regarding each objective. 

Problem Number Three Solved 

Since both the objectives and the test measures employed are drawn 
from the same agency, comparisons of preference and current status data are 
rendered far more simple. We would only have to calculate the average preference 
estimates for each of the involved groups, then identify the percent of students 
mastering the objectives. These comparisons could be presented in summary form 
such as seen in Figure 2. 
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Figure 2. 

It is now necessary, of course, to make those difficult decisions re- 
garding which objectives, among competing alternatives, reflect the most important 
educational needs. While we do not yet have a simple computer program which could 
take such data and pump out a precise identification of the most crucial student 
needs, a careful appraisal of such information should yield far more enlightened 
choices among alternative needs than by using currently extant schemes. 

Person and Item Sampling 

In using the procedure recommended here, it is important to use sampling 
procedures judiciously throughout, both with respect to sampling the people in- 
volved and to the items to which they are responding. For purposes of economy, 
we would undoubtedly wish to select only a sample of individuals from the several 
clienteles involved. For instance, if we wished to use teacher groups in a 
State-wide assessment, then we ought to employ sampling procedures so that we do 
not burden all teachers in the State with the task of rating objectives. Simi- 
larly, samples of student? would need to be chosen for testing purposes. 

While the value of such person sampling is generally recognized, the 
utility of item sampling has not beer comprehended by most needs assessment per- 
sonnel. Item sampling permits one to administer different items to different 
people to obtain a group estimate. In other words, if I am testing one hundred 
students and wish to know how the one hundred students perform on tour objectives, 
I might constitute a test of four separate parts, each part reflecting one 
separate objective, then randomly administer each of the different sections to 
only twenty-five students. Since a sizable portion of the class would be re- 
sponding to each test item, I could secure a perfectly adequate indication of 
the degree to which students could master those items. 

Similarly, w.; need not subject parents to the necessity of rating 
hundreds of objectives. We could, instead, put together a variety of different 
short sets of objectives and let randomly selected parents rate anywhere from 
ten to twenty objectives, thereby taking no more than a few minutes of each 
parent’s time. The use of person sampling and item sampling procedures is re- 
quisite in the economic implementation of this approach to needs assessment. 




57 



Problem Number Four : Insufficien t Collections of Objectives- Items 



Unfortunately, at the moment, this approach to needs assessment can 
only be implemented on a far smaller basis than is desirable. The reason for 
this restriction is that we currently do not have sufficiently large collections 
of different objectives and items to permit its wide-scale implementation. 

While there are some collections, primarily available through the Instructional 
Objectives Exchange, we need many more such collections of objectives and items. 
Here's where all educators, your group in particular, could be of considerable 
assistance . 



You can help provide those object ives-item collections in two ways. 
First, by contributing any objec tives-item pools not already possessed by IOX so 
that they can be made available to those wishing to perform this kind of educa- 
tional needs assessment. Secondly, by supporting the development of objectives- 
item pools of particular interest to your needs assessment operation.. The tech- 
nical procedures for developing objectives and items exist. The only limitation 
in developing such collections is the limited resources now available in the 
Instructional Objectives Exchange. On a contract basis, additional pools of ob- 
jectives and items could be provided and shared in hard-to-measure areas. 

A Self-Correctin g System 

While not all of the problems to be faced in implementing this approach 
to educational needs assessment systems have been identified, for it has not been 
used on a wide-scale basis, certainly the strategy is sufficiently manageable to 
permit improvement as it is employed. 

Through the assembly of collections of instructional objectives and 
related criterion measures, coupled with the efforts of those who would systemati- 
cally determine educational needs, we will surely ir> crease the quality of our edu- 
cational needs assessment operations so that we can identify the educational 
objectives we really ought to be pursuing. 
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The evaluation of the three regional Title III ESEA workshops on "Fa- 
cilitating the Creative Educational Change Process Through Effective Management" 
is reported in two parts: Part I -Procedure for Evaluation and Part II - Sum- 

mary of the Results of Evaluation. 

PART I - PROCEDURE FOR EVALUATION 

All three regional workshops had the same goals. The goals were: 

1. To increase understanding and competency in developing 
strategies and techniques in the assessment of educa- 
tional needs. 

2. To provide orientation for new Title III Coordinators 
and for State Advisory Council members. 

3. To increase understanding and competency in developing 
strategies and techniques in educational accountability. 

4. To provide an opportunity to discuss individual State 
problems • 

5. To increase understanding and competency in developing 
strategies and techniques in dissemination and evaluation. 

6. To assist State educational agencies in the development of 
strategies in the areas of staff development, educational 
needs assessment, evaluation, and dissemination to complete 

* the FY 1970 Title III State Plan requirements. 

To determine the degree to which these goals were achieved, two types 
of evaluation instruments were developed: (1) A pre- and post-conference test 

to measure individual and group changes in attitudes, knowledge, and understand- 
ing of certain components in Title III program management; and (2) a post- con- 
ference questionnaire to ascertain conferees 1 opinions of workshop strengths and 
weaknesses * 



At each of the regional meetings, the pre-test was given on the first 
day and the post-test on the last day. The same instrument was used for the pre- 
and post-test. The tests were identified by code number so that individual 
changes between tests could be determined. The post-conference questionnaires 
were distributed and collected on the last day. A total of 178 State Title III 
staff members, chairmen of State Advisory Councils, members of USOE Title III 
staff, and other State and local educational agency personnel participated in 
the five-day conferences. 

Not all the participants remained for the full five days. Some of the 
States sent one team of representatives the first two or three days and re- 
placed it with another team for the rest of the week because it was difficult 
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to have any one staff member remain away from his office for the full five days 
or because they wanted as many staff members exposed to this conference as 
possible, even if to only half of the conference. 

In addition to the data gathered by the pre- and post-test and the 
post-conference questionnaire, valuable data concerning the strengths and weak- 
nesses of the conference were gathered through careful assessment of the reports 
that were written and submitted by the small group workshops. 

PART II “ SUMMARY OF THE RESULTS OF EVALUATION 

"This is the best USOE conference I've attended!" ... "The conference 
reinforced ideas I already had." ... "No, I believe that the objectives of the 
Thursday program were not met." Such were the sentiments expressed by the con- 
ferees. Other results of the evaluation of the three regional conferences are 
summarised under the six general goals of the conferences. 

Conference Goal 1. To increase understanding and competency in 

developing strategies and techniques in the assessment of 

educational needs. 

When the conferees were asked, "Do you feel that the conference suc- 
ceeded in helping you assess your State's educational needs?", about sixty-eight 
percent of the ninety- four respondents said, "yes," and thirty-two percent 
answered, "no." Nevertheless, when the same conferees were asked, "In con- 
nection with which administrative area do you feel that the conference was most 
successful?" the largest number of persons, sixty-nine, or about seventy-four 
percent of the respondents, answered, "Needs Assessment." Perhaps some of the 
twenty-six participants who felt that the conference did not succeed in achieve - 
ing this objective expected the conference to give more specific examp3.es of 
the "hows" instead of the "whys" of conducting State-wide needs assessment. 

Most conferees agreed that needs assessments should focus on the 
learner instead of on instructional resources. About eighty-three percent of 
the respondents in the pre-test and about ninety-four percent in the post-test 
agreed that focusing of needs assessments on instructional needs such as numbers 
of teachers, school facilities, and instructional materials is not sufficient to 
bring about progress in meeting learner needs. 

Almost all conferees believed that State-wide achievement testing can 
help identify learner needs in the cognitive domain but only forty percent of 
the respondents in the pre-test and fifty-five percent in the post-test believed 
that needs assessment as generally practiced can identify learner needs in the 
affective domain. 

Conference Goal 2. To provide orientation for new Title III 

coordinators and for St^te advisory council members. 

A large portion of the first day of the conference was spent in general 
orientation of new Title III State coordinators and State advisory council mem- 
bers by members of the. State Plans for Educational Innovation Branch on the 
status and changes in the ESEA Title III program. 

When the conferees were asked, "Did the conference clarify questions 
you may have had regarding regulations, procedures, etc., in connection with the 



administration of Title III? 11 , about eighty-eight percent of the ninety-four re- 
spondents said, "yes." 

This is the conference goal in which the pre- and post-tests showed the 
highest degree of success. Only sixty-two percent of the sixty-two respondents 
in the pre-test, but ninety-four percent in the post-test, were able to identify 
the amount of funds appropriated for ESEA Title III in Fiscal Year 1969. 

Only sixty-five percent of the respondents in the pre-test, but ninety- 
five percent in the post-test, knew that the State advisory councils send their 
annual reports to the National Advisory Council and to Congress through the State 
education agency. 

Only thirty-seven percent of the respondents in the pre-test, but 
ninety-three percent in the post-test, knew that the CPIR has replaced the Title 
III Statistical part of the State Annual Report form. 

Conference Goal 3. To increase understanding and competency in 

developing strategies and techniques in educational accounta- 
bility. 

About sixty-two percent of the ninety-four respondents to the post- 
conference questionnaire were favorable to the concept of educational accounta- 
bility; twenty-one percent were either skeptical or believed the concept to be 
vague. The remaining respondents had reservations on certain p£\rts of the gen- 
eral concept such as, for example, the provision for penalties. When the re- 
spondents were asked what kind of help the U. S. Office of Education might give 
to bring about educational accountability, a large number suggested providing 
more information or conducting regional training sessions on accountability. 

About seventy-one percent of the ninety-four respondents said the con- 
ference was helpful in giving them guidance regarding strategies and techniques 
to be employed in bringing about educational accountability. About seventeen 
percent said the conference was partially helpful and twelve percent said it 
was not very helpful. 

The pre- and post-tests showed some progress in the area of accounta- 
bility. Only fifty- two percent of the respondents in the pre-test but eighty- 
four percent in the post-test were able to define Educational accountability 11 
correctly. About ninety percent of the respondents in the pre-test but ninety- 
seven percent in the post-test defined "performance objectives" correctly. 

About eighty-six percent of the respondents in the pre-test but one hundred per- 
cent in the post- test were able to define "performance criteria 11 correctly. 

Only forty-eight percent of the respondents in the pre-test and fifty- 
five percent in the post-test agreed with this statement; "A performance con- 
tract requiring specified results in a specified time and budget with rewards 
or penalties according to results delivered is desirable." Nevertheless, a 
little over three-fourths of the respondents believed that the concept of account- 
ability for educational results can be applied to the Title III program by the 
States. The pre- and post-test reactions to the desirability of educational 
accountability is interesting. Thirteen persons who believed in the desirability 
of accountability in education in the pre-test changed their minds in the post- 
test. On the other hand, nineteen persons who did not believe in the desira- 
bility of this concept expressed agreement in the post-test. 

3 



Conference Goal 4. To provide an opportunity to discuss 
individual State problems. 

Time and place was set aside for small group or individual consulta- 
tions with the speakers or with USOE or SEA staff members each day. Respondents 
indicated in the post-conference questionnaire that this was ,f the first opportunity 
they had had to meet advisory council members from other States," that "time set 
aside for sharing experience" ras profitable. About thirty percent of the re- 
spondents said that "miscella tous" experiences of the conference were as impor- 
tant as the structured program. 

Conference Goal 5. To increase understanding and competency in 
developing strategies and techniques in dissemination and 
evaluation. 



About eighty-six percent of the respondents in the pre-test and about 
ninety-seven percent in the post-test agreed that State educational agencies must 
develop procedures for evaluating projects and bringing about tne adaptation and 
adoption of promising practices in Title III programs. 

It is interesting to note that in the pre-test fourteen persons >ut of 
the sixty-two respondents disagreed that States have a responsibility to commit 
State resources to the adaptation and installation of innovation in local schools. 
At the end of the conference, every respondent agreed with this idea. 

ESEA Title III is considered a cutting edge in the change process in 
education. But only forty-four percent of the ninety-four respondents thought 
that the conference gave them a clear understanding of what constituted effective 
management leading to the creation of educational change in the State departments. 
The largest number of negative responses, twenty-six percent, wanted more infor- 
mation that would give them a clearer understanding; another four percent said 
not enough time had been spent on the subject. A small number, four percent, 
said application of the concept was questionable. Nevertheless, the percent of 
respondents who believe they can have considerable influence upon changing edu- 
cational programs in their own State departments went up from fifty-two percent 
in the pre-test to seventy-three percent in the post-test. 

In Summary 

The conferees were asked, "Generally, overall, how would you rate this 
conference?" They were asked to check a point on a ten-point scale, with one 
being "poor" and ten being "excellent." About sixty- three percent of the ninety- 
four respondents rated the workshop eight or higher. 



Rating Percent 

Scale Response 



5 

6 

7 

8 
9 

10 

Total response 94 



0 % 

3% 

11 % 

23% 

44% 

13% 

6 % 

Total percent 1007. 
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Workshop Participants 



STATE 





CSSO 


SAC 

CHAIRMAN 


SAC 

MEMBERS 


TITLE III 
COORDINATOR 


SEA 

STAFF 


OTHERS 


TOTALS 


1 

Alabama 








X 


2 




3 


Alaska 




X 




X 






2 


Arizona 


X 


X 




X 


3 




5 


.Arkansas 








X 






1 


California 




X 




X 


2 


X 


5 


Colorado 




X 




X 






2 


Connecticut 








X 

1 






X 


Delaware 








X 






1 


Florida 




X 




X 


6 




8 


Georgia 




X 




X 


X 




3 


Hawaii 




X 




X 


X 




0 


Idaho 




X 




X 






2 


Illinois 










2 




2 


Indiana 








X 






1 


Iowa 








X 


X 




2 


Kansas 




X 




X 






2 


Kentucky 








X 






1 


Louisiana 




X 


X 


X 


X 




4 


Maine 








X 


1 




1 


Maryland 








X 


1 

1 3 




4 


Massachusetts 




1 


X 


x 


1 x 
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STATE 





CSSO 


SAC 

CHAIRMAN 


SAC 

MEMBERS 


TITLE III 
COORDINATOR 


SEA 

STAFF 


OTHERS 


TOTALS 


Michigan 








I 

X 






1 


Minnesota 








X 


2 




3 


Mississippi 




X 




X 




X 


3 


Missouri 




X 




X 


X 




3 


Montana 




X 




X 


X 




3 


Nebraska 




X 




X 






2 


Nevada 




X 




X 






2 


New Hampshire 






X 


X 


X 




3 


New Jersey 










X 




1 


New Mexico 








X 






1 


New York 








X 






1 


North Carolina 




X 




X 


X 




3 


North Dakota 




X 




X 






2 


Ohio 




X 




X 






2 


Oklahoma 




X 




X 






2 


Oregon 




X 




X 






2 


Pennsylvania 






3 


X 


6 


X 


11 


Rhode Island 




X 




X 


X 




3 


South Carolina 








X 


X 




2 


South Dakota 




X 




X 


2 




4 


Tennessee 








X 






1 


Te.raB 




X 




X 


X 




3 


Utah 




X 




X 


X 




3 


Vermont 






X 








1 


Virginia 






X 


X 


X 




3 


Washington 








X 






1 
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